In the days of yore, when GNOME was just version 2 point something, it used to be possible to rip out (most of) GNOME with a simple apt-get remove libgnome2-common. This would leave a few things lying around, but not enough for most people to worry about.
I have no idea how to do the analogous operation with GNOME 3.x.
