I realized if I re-build numpy it takes the interpreter compile options or you
can edit site.cfg.
I actually solved the issue by getting the newest svn copy of numpy. It must be
something with 1.6b2. The newest SVN has no issues with either library and
calls to sum or prod. I was getting memory leak with both the tcmalloc and
malloc with the 1.6b2 version of numpy.
There are a few places where we (improperly) directly call malloc()
instead of PyMem_Malloc(), so yes, you should rebuild numpy against
TCMalloc in addition to the Python interpreter.
How do I build against the -ltcmalloc library? Just not sure how to
input compile time arguments with the build script.
from numpy import arange, sum
for x in range(10000000):
inhibVal = sum(arange(15))
Memory usage stays constant with Ubuntu 11.04, 64-bit, using the numpy
1.5.1 package from ubuntu, and using 1.6.1.dev-a265004.
efiring@manini:~$ uname -a
Linux manini 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24 UTC
2011 x86_64 x86_64 x86_64 GNU/Linux
Eric
Also noticed that the same issue occurs as described with using the prod
instead of sum. Other commands like copy and reshape have no issues. I
have a machine with 11.04 on it so I will give that a go later to see
what goes.
Thanks, Jeff
Message: 2 Date: Sun, 22 May 2011 12:32:43 -0500 From: Robert Kern
<[email protected]> Subject: Re: [Numpy-discussion] Python memory
management issues using Linux. Maybe Numpy related. To: Discussion of
Numerical Python <[email protected]> Message-ID:
<[email protected]> Content-Type:
text/plain; charset=UTF-8 On Sat, May 21, 2011 at 23:27, Jeffrey Spencer
<[email protected]> wrote:
> I have had issues with Linux (Ubuntu 10.04 x86_64) not releasing memory
> back to the OS. I have code here the exhibits the issue:
>
>
http://stackoverflow.com/questions/5975255/memory-allocated-to-python-in-the-os-is-never-released-back-in-linux-even-after-g
>
> Then I read athttp://pushingtheweb.com/2010/06/python-and-tcmalloc/
> that this was a problem fixed by compiling your own version of Python
> using the TCMalloc library which I have now done. The issue is still
> exhibited so I was keen to know if possibly it was because numpy is
> compiled and linked to the default memory management library (not sure
> if this is true or how this works but would I also need to compile numpy
> linked to the TCMalloc library). Let me know if anyone has any insight
> on this issue. Or if it isn't related to numpy at all.
There are a few places where we (improperly) directly call malloc()
instead of PyMem_Malloc(), so yes, you should rebuild numpy against
TCMalloc in addition to the Python interpreter.
-- Robert Kern "I have come to believe that the whole world is an
enigma, a harmless enigma that is made terrible by our own mad attempt
to interpret it as though it had an underlying truth." ? -- Umberto Eco
--
________________________
Jeffrey Spencer
[email protected]
_______________________________________________
NumPy-Discussion mailing list
[email protected]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
