Old is pretty much correct. I just self published my first book using a pen name, so I know a little bit about it.
You can write using a pen name; you will have to conduct business using your real name (to coordinate with your bank, paypal, etc...). You can set up an LLC (incorporated in a state like New Mexico or Delaware- known for their privacy laws and can be established by people who do not live in those states), which can help. The LLC could handle the business side of things to help add another layer of anonymity.
Another issue that I'm finding so far in my small amount of time is the use of a postal code. You can find a company that will handle your distribution, but many self publishers like to handle their own (mailing out books from home). It can help to make sure you're getting the correct amount of sales, among other things. This means a potential stalker could trace the address you place on the envelope, or trace the post office stamp back to the original office.
If you're selling ebooks, I'm sure there is a way to trace the IP address so even that is not a total protection. You could use something like TOR to mask your IP address.
Regarding taxes, I'm not sure.
The best way I've found to protect my identity in my small time running my store is to act like a customer who wants to buy my books, and scrutinize every detail of dealing with my business to look for holes. For example, when I first started using my business email, I had a generic "
[email protected]" address. However, when I would send emails using that name, it included my real name with the email. I had to change around my email preferences so that my real name could not be seen. Or I should say, not seen as easily.
Honestly though, if you use a pen name, you're headed in the right direction. Someone who is going to jump through a bunch of hoops to find you will do it by any way possible, whether you have a pen name or not.