On Tuesday 23 February 2010, Alec Matusis wrote: > When I start the process, both python object sizes and their counts rise > proportionally to the numbers of reconnected clients, and then they > stabilize after all clients have reconnected. > At that moment, the "external" RSS process size is about 260MB. The > "internal size" of all python objects reported by Heapy is about 150MB. > After two days, the internal sizes/counts stay the same, but the external > size grows to 1500MB. > > Python object counts/total sizes are measured from the manhole. > Is this sufficient to conclude that this is a C memory leak in one of the > external modules or in the Python interpreter itself?
In general, there are other reasons why heap size and RSS size do not match: 1. pages are empty but not returned to the OS 2. pages cannot be returned to the OS because they are not completely empty It seems Python has different allocators for small and large objects: http://www.mail-archive.com/python-l...@python.org/msg256116.html http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete- a-large-object.htm Assuming Python uses malloc for all its allocations (does it?), it is the malloc implementation that determines whether empty pages are returned to the OS. Under Linux with glibc (your system?), empty pages are returned, so there reason 1 does not apply. Depending on the allocation behaviour of Python, the pages may not be empty though, so reason 2 is a likely suspect. Python extensions written in C could also leak or fragment memory. Are you using any extensions that are not pure Python? Bye, Maarten _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
