On Sun, 2012-05-06 at 01:42 +0200, Vlad K. wrote: > As I understand it Python won't release internally released memory back > to OS.
Glad to read that you have an acceptable solution. The issue piqued my curiosity and I found this from June 2010: "we have recently experienced problems with Python not giving back memory to the OS in Linux. It reuses allocated memory internally, but never releases free memory back to the OS": http://pushingtheweb.com/2010/06/python-and-tcmalloc/ in which a comment points to a Python 3.3 (alpha 01) hg commit that fixes the reported problem: http://hg.python.org/cpython/rev/f8a697bc3ca8 The discussion on the issue ticket associated with the commit: http://bugs.python.org/issue11849 contains further details, including an example of exactly how creating large XML files in Python might cause the memory retention via lingering refs --- just as Ben describes. AIUI, the standard approach to handling large XML files is to use a stream processor such as SAX. finally: self.curiosity.piqued = False Cheers, -- Graham Higgins http://bel-epa.com/gjh/
signature.asc
Description: This is a digitally signed message part
