MKoool wrote: > I have an application with one function called "compute", which given a > filename, goes through that file and performs various statistical > analyses. It uses arrays extensively and loops alot. it prints the > results of it's statistical significance tests to standard out. Since > the compute function returns and I think no variables of global scope > are being used, I would think that when it does, all memory returns > back to the operating system. > > Instead, what I see is that every iteration uses several megs more. > For example, python uses 52 megs when starting out, it goes through > several iterations and I'm suddenly using more than 500 megs of ram. > > Does anyone have any pointers on how to figure out what I'm doing > wrong? >
if gc.collect() doesn't help: maybe objects of extension libs are not freed correctly. And Python has a real skeleton in the cupboard: a known problem with python object/libs when classes with __del__ are involved ( Once suffered myself from such tremendous "unexplainable" memory blow up until I found this "del gc.garbage[:]" remedy: <http://www.python.org/doc/current/lib/module-gc.html> garbage A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects). By default, this list contains only objects with __del__() methods.3.1Objects that have __del__() methods and are part of a reference cycle cause the entire reference cycle to be uncollectable, including objects not necessarily in the cycle but reachable only from it. Python doesn't collect such cycles automatically because, in general, it isn't possible for Python to guess a safe order in which to run the __del__() methods. If you know a safe order, you can force the issue by examining the garbage list, and explicitly breaking cycles due to your objects within the list. Note that these objects are kept alive even so by virtue of being in the garbage list, so they should be removed from garbage too. For example, after breaking cycles, do del gc.garbage[:] to empty the list. It's generally better to avoid the issue by not creating cycles containing objects with __del__() methods, and garbage can be examined in that case to verify that no such cycles are being created. Robert -- http://mail.python.org/mailman/listinfo/python-list
