Thanks all for responses. Continuation below: 2009/4/18 Charles R Harris <charlesr.har...@gmail.com>: > > > On Fri, Apr 17, 2009 at 9:25 AM, Dan S <dan.s.towell+nu...@gmail.com> wrote: >> >> Hi - >> >> I have written a numpy extension which works fine but has a memory >> leak. It takes a single array argument and returns a single scalar. >> After reducing the code down in order to chase the problem, I have the >> following: >> >> static PyObject * kdpee_pycall(PyObject *self, PyObject *args) >> { >> PyObject *input; >> PyArrayObject *array; >> int n, numdims; >> >> if (!PyArg_ParseTuple(args, "O", &input)) >> return NULL; >> >> // Ensure we have contiguous, 2D, floating-point data: >> array = (PyArrayObject*) PyArray_ContiguousFromObject(input, >> PyArray_DOUBLE, 2, 2); >> >> if(array==NULL){ >> printf("kdpee_py: nullness!\n"); >> return NULL; >> } >> PyArray_XDECREF(array); >> Py_DECREF(array); // destroy the contig array >> Py_DECREF(input); // is this needed? doc says no, but it seems to fix > ^^^^^^^^^^^^^^^^^^^^ > > Shouldn't be, but there might be a bug somewhere which causes the reference > count of input to be double incremented. Does the reference count in the > test script increase without this line?
Yes - if I comment out Py_DECREF(input), then sys.getrefcount(a) goes from 2, to 5002, to 10002, every time I run that little 5000-fold test iteration. (Py_DECREF(input) is not suggested in the docs - I tried it in desperation and for some reason it seems to reduce the problem, since it gets rid of the refleak, although it doesn't solve the memleak.) >> return PyFloat_FromDouble(3.1415927); // temporary >> } >> >> >> The test code is >> >> from numpy import * >> from kdpee import * >> import sys >> a = array( [[1,2,3,4,5,1,2,3,4,5], [6,7,8,9,0,6,7,8,9,0]]) >> sys.getrefcount(a) >> for i in range(5000): tmp = kdpee(a) >> >> sys.getrefcount(a) >> >> Every time I run this code I get "2" back from both calls to >> sys.getrefcount(a), but I'm still getting a memory leak of about 0.4 >> MB. What am I doing wrong? > > So about 80 bytes/iteration? How accurate is that .4 MB? does it change with > the size of the input array? OK, pinning it down more precisely I get around 86--90 bytes per iteration (using ps, which gives me Kb resolution). It does not change if I double or quadruple the size of the input array. Today I tried using "heapy": using the heap() and heapu() methods, I find no evidence of anything suspicious - there is no garbage left lying on the heap of python objects. So it's presumably something deeper. But as you can see, my C code doesn't perform any malloc() or suchlike, so I'm stumped. I'd be grateful for any further thoughts. Thanks Dan _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion