Rhythmbox hides its GUI in the notification area when minimized to get out of the way. To exit, you should press quit or stop music and then close the window. If you don't like this behaviour, you can install Audacious, a Winamp clone for GNU/LInux. It will just exit if you click on the x button of the window title bar.
sudo apt-get audacious
