help with memory leak

2014-05-27 Thread Neal Becker
I'm trying to track down a memory leak in a fairly large code.  It uses a lot 
of 
numpy, and a bit of c++-wrapped code.  I don't yet know if the leak is purely 
python or is caused by the c++ modules.

At each iteration of the main loop, I call gc.collect()
If I then look at gc.garbage, it is empty.

I've tried using objgraph.  I don't know how to interpret the result.  I don't 
know if this is the main leakage, but I see that each iteration there are more
'Burst' objects.  If I look at backrefs to them using this code:

   for frame in count(1): ## main loop starts here
gc.collect()
objs = objgraph.by_type('Burst')
print(objs)
if len (objs) != 0:
print(objs[0], gc.is_tracked (objs[0]))
objgraph.show_backrefs(objs[0], max_depth=10, refcounts=True)

I will get a graph like that attached

A couple of strange things.

The refcounts (9) of the Burst object don't match the number of arrows into it.
There are 2 lists with 0 refs.  Why weren't they collected?-- 
https://mail.python.org/mailman/listinfo/python-list


Re: help with memory leak

2014-05-27 Thread Chris Angelico
On Wed, May 28, 2014 at 5:56 AM, Neal Becker ndbeck...@gmail.com wrote:
 I'm trying to track down a memory leak in a fairly large code.  It uses a lot 
 of
 numpy, and a bit of c++-wrapped code.  I don't yet know if the leak is purely
 python or is caused by the c++ modules.

Something to try, which would separate the two types of leak: Run your
program in a separate namespace of some sort (eg a function), make
sure all your globals have been cleaned up, run a gc collection, and
then see if you still have a whole lot more junk around. If that
cleans everything up, it's some sort of refloop; if it doesn't, it's
either a global you didn't find, or a C-level refleak.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list