I agree with all of Travis' comments below and committed the suggested
changes to numarray CVS. I found one other numarray change needed
for Francesc's examples to run (apparently) leak-free:
Py_INCREF(obj)
Py_XDECREF(a->base)
a->base = obj
Py_DECREF(cobj)
Thanks Travis!
Regards,
Todd
Travis Oliphant wrote:
> Francesc Altet wrote:
>
>> Hi,
>>
>> I was tracking down a memory leak in PyTables and it boiled down to a
>> problem
>> in the array protocol. The issue is easily exposed by:
>>
>> for i in range(1000000):
>> numarray.array(numpy.zeros(dtype=numpy.float64, shape=3))
>>
>> and looking at the memory consumption of the process. The same happens with:
>>
>> for i in range(1000000):
>> numarray.asarray(numpy.zeros(dtype=numpy.float64, shape=3))
>>
>> However, the numpy<--numarray sense seems to work well.
>>
>> for i in range(1000000):
>> numpy.array(numarray.zeros(type="Float64", shape=3))
>>
>> Using numarray 1.5.1 and numpy 1.0b1
>>
>> I think this is a relatively important problem, because it somewhat prevents
>> a
>> smooth transition from numarray to NumPy.
>>
>>
>>
>
> I tracked the leak to the numarray function
>
> NA_FromDimsStridesDescrAndData
>
> This function calls NA_NewAllFromBuffer with a brand-new buffer object
> when data is passed in (like in the case with the array protocol). That
> function then takes a reference to the buffer object but then the
> calling function never releases the reference it already holds. This
> creates the leak.
>
> I added the line
>
> if (data) {Py_DECREF(buf);}
>
> right after the call to NA_NewAllFromBuffer and the leak disappeared.
>
> For what it's worth, I also think the base object for the new numarray
> object should be the object passed in and not the C-object that is
> created from it.
>
> In other words in the NA_FromArrayStruct function
>
> a->base = cobj
>
> should be replaced with
>
> Py_INCREF(obj)
> a->base = obj
> Py_DECREF(cobj)
>
>
> Best,
>
>
> -Travis
>
>
>
>
>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/numpy-discussion
