I am having trouble identifying the source of a memory leak in a Windows Python program. The basic gist is as follows: 1. Generate a directed graph (approx. 1000 nodes). 2. Write the graph to a file. 3. Use the os.system command to invoke another program which processes the graph file (graphViz), and generates a gif image of the graph. 4. Use another os.system command to delete the intermediate file, which is no longer needed. 5. Append to a global list variable some summary information about the graph (a few floats, a few ints, and a couple of short strings). 6. Repeat the above steps several thousand times. 7. Sort the global summary list. 8. Write the summary information out to a file.
When running this program, Python consumes all memory after about a thousand graphs. I have confirmed the following: 1. I am closing all files that I open. 2. Although within the directed graph, nodes refer back and forth to each other, the main object which manages the directed graph is not part of anything cyclical. It's reference count is in fact going to zero when the graph goes out of scope. 3. After running gc.collect(), I have confirmed that the nodes are not in the garbage list. I interpret this to mean that although their may be cyclical references among them, the collector was able to determine that they were unreachable as a group, and successfully collected them. 4. I am generating all these graphs in a for loop (using xrange), not through a recursive process that would cause the stack to grow. So really the only thing that should be growing here is the global list that maintains summary information, and since I'm only adding little bits of data, it's hard to imagine how this could be exhausting 2GB of memory after just a thousand iterations. Can anyone help me think of other possible sources of a leak? Are there any classic "python-gotchas" I'm missing here, such as something leaking from calling os.system so many times? Any other suggestions as to what strategies I could employ to track down the leak? Thanks, Mark -- http://mail.python.org/mailman/listinfo/python-list
