I know of at least one modern-day smartphone (that is considered one of the
best around, and that uses Android), that lets you (easily) take off the
battery, like with an old-day mobile phone:
http://www.youtube.com/watch?v=xBIEkbjs8Ms
It has a pretty good price/quality relationship. And, because of that option
(of letting us take off the battery, when we don't want to be tracked and
listened to), I've even considered buying one, myself.
But, with cameras on the front and the back (and me not knowing if someone
might be taking a peek), it's too much for me... ;)