On Apr 10, 2008, at 9:51 AM, Brian Granger wrote:
>
> Hi,
>
> (dual posted to sage and cython)
>
> A few of us (ipython and mpi4py devs) are wondering what the
> best/safest way of allocating dynamic memory in a local scope
> (method/function) is when using cython. An example would be if you
> need an array of c ints that is locally scoped.
>
> The big question is how to make sure that the memory gets freed - even
> if something goes wrong in the function/method. That is, you want to
> prevent memory leaks. It looks like in sage, the
> sage_malloc/sage_free functions are used for this purpose:
>
> from sage/graphs/graph_isom.pyx:
>
> 176 def incorporate_permutation(self, gamma):
> 202 cdef int *_gamma = <int *> sage_malloc( n * sizeof
> (int) )
> 203 if not _gamma:
> 204 raise MemoryError("Error allocating memory.")
> 205 for k from 0 <= k < n:
> 206 _gamma[k] = gamma[k]
> 207 self._incorporate_permutation(_gamma, n)
> 208 sage_free(_gamma)
>
> Because sage_malloc is #defined to malloc in stdsage.h, I think there
> is a significant potential for memory leaks in code like this. Are we
> thinking correctly on this issue? Isn't this a huge problem?
>
> Lisandro Dalcin (author of mpi4py) came up with the following trick
> that, while more complicated, prevents memory leaks:
>
> cdef extern from "Python.h":
> object PyString_FromStringAndSize(char*,Py_ssize_t)
> char* PyString_AS_STRING(object)
>
> cdef inline object pyalloc_i(int size, int **i):
> if size < 0: size = 0
> cdef Py_ssize_t n = size * sizeof(int)
> cdef object ob = PyString_FromStringAndSize(NULL, n)
> i[0] = <int*> PyString_AS_STRING(ob)
> return ob
>
> and now
>
> def foo(sequence):
> cdef int size = len(sequence),
> cdef int *buf = NULL
> cdef object tmp = pyalloc_i(size, &buf)
>
> This could probably be adapted into a malloc-like function. What do
> people think?
This is actually the same solution that popped into my mind when I
first read your question, and I think it is a very good one. Using
string objects is particularly interesting because the they are
highly optimized (for example, the buffer allocation is done as part
of the object allocation). I would probably have it take a void**
rather than make an allocator specific to ints. If people like this
idea, I could add such a function to Sage. Maybe it would even be
worthwhile adding it to Cython (in one of the included header files).
Alternatively, using a try-finally will work. If the buffers all are
set to start as NULL then one just frees the non-NULL buffers at the
end (so one doesn't have to keep track of where in the procedure
things went wrong).
- Robert
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
