I suspect that in fact most of us agree on this point and have not been realizing it.
I don't think a baker should be able to say "I will not sell you a wedding cake because you are gay." (With caveats I've already mentioned.)
I do think a baker should be able to say "I will not write 'Matt & Geoff' on your wedding cake," even if the baker would write "Matt & Geraldine."
My personal feelings aren't covered by the law, but I did believe that the law would protect certain things written on the cakes, like 'Matt & Geoff' (in most states), yes. Same with 'Happy Confirmation' and other phrases that have to do with protected class status. But that article by the lawyer linked a couple of times casts doubt on that. Of course, I mean for a bakery that offers custom writing.
My personal feelings are that sole proprietorships and partnerships that use SSNs instead of federal tax ID numbers that have sales under a certain (small) amount should have the right to refuse service or products to anyone. I'd except rental contracts and the like, of course, and if they were the only seller within a certain radius, too.
Then, larger, more public (as to ownership) companies would have to write or serve pretty much anything, adding a freedom of speech kind of idea in there. Serving the public means the good, bad and ugly, imho, at that level of sales.
I'd except artistic endeavors like photographers (and sculptors, writers, etc). Their freedom of expression needs to be preserved, even though they do it for a living for the public, imho. Most of them probably don't fit the business criteria I mentioned anyway, but there are some large companies, too, so the artistic expression part needs to be spelled out, I think.