Message: 3 Date: Sun, 22 May 2011 09:32:10 -0500 From: Bruce Southey<[email protected]> Subject: Re: [Numpy-discussion] Python memory management issues using Linux. Maybe Numpy, related. To: Discussion of Numerical Python<[email protected]> Message-ID:<[email protected]> Content-Type: text/plain; charset=ISO-8859-1
On Sun, May 22, 2011 at 8:42 AM, Ralf Gommers <[email protected]> wrote:
> > > On Sun, May 22, 2011 at 2:54 PM, Jeffrey Spencer<[email protected]> > wrote:>> >> Sorry. I attached here the class and script to run it. It is just a >> minimalistic example of what I'm really doing that exhibits the same >> behavior. All the script does is run at.createSpecific after instantiating >> the class. I have seen this exhibited on many other cases but it runs fine >> in Windows XP. Here is what the script does: >> >> import MemoryLeak >> at = MemoryLeak.Model() >> at.createSpecific() >>> That manages to eat up all my cpu cycles for about 5 minutes, but no leak > (on OS X). I also don't see anything in the code that can potentially give > problems. I suggest you strip this down a lot more so you can isolate the > problem. > > If in your original code you were making a lot of figures in one of those > for loops without explicitly closing them, that could perhaps be a problem. > But you removed all I/O and matplotlib related code already (except for > imports and some keywords). > > Ralf > > _______________________________________________ > NumPy-Discussion mailing list > [email protected] > http://mail.scipy.org/mailman/listinfo/numpy-discussion >
A good OS is likely to keep Python module in memory just because it knows that it might be requested again. However, from the little I know (too much LWN), if Linux needs that memory it will reclaim that memory only when it needs it or part of some other process. Measuring memory usage using simple tools (like ps or top) are very far from sufficient, so exactly how do you know that the memory has not been released? Bruce Bruce, The reason I have the issue is it ends up swapping memory over to the swap. After this, the whole computer runs at a crawl. Even if as your saying it is allocated as free just not reclaimed from Python. The OS should try to get the memory back before swapping everything. I even have my swappiness set lower to avoid sending things to the swap. Jeff -- ________________________ Jeffrey Spencer [email protected]
_______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
