If you could attach the output generated from running valgrind (see https://wiki.ubuntu.com/Valgrind) when you start rhythmbox to when you see the memory usage change significantly, that may be of some help once you've narrowed down how you're using rhythmbox.
-- Possible memory leak https://launchpad.net/bugs/58372 -- desktop-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/desktop-bugs
