The way I read the actual text it suggests that following and then unfollowing in a short period of time is the violation...not that you can't unfollow someone who didn't follow you. In fact, isn't following someone who doesn't want to follow you back a form of stalking for some people? I've always taken it as a given that it's a courtesy to not continue following someone indefinitely if they're not following back. There are exceptions of course depending on the person or topic and how that person is operating their account and whether they are a public or private entity.
http://twitter.zendesk.com/forums/26257/entries/18311
