On Thu, Apr 10, 2008 at 9:51 AM, Brian Granger <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> (dual posted to sage and cython)
>
> A few of us (ipython and mpi4py devs) are wondering what the
> best/safest way of allocating dynamic memory in a local scope
> (method/function) is when using cython. An example would be if you
> need an array of c ints that is locally scoped.
>
> The big question is how to make sure that the memory gets freed - even
> if something goes wrong in the function/method. That is, you want to
> prevent memory leaks. It looks like in sage, the
> sage_malloc/sage_free functions are used for this purpose:
>
> from sage/graphs/graph_isom.pyx:
>
> 176 def incorporate_permutation(self, gamma):
> 202 cdef int *_gamma = <int *> sage_malloc( n * sizeof(int) )
> 203 if not _gamma:
> 204 raise MemoryError("Error allocating memory.")
> 205 for k from 0 <= k < n:
> 206 _gamma[k] = gamma[k]
> 207 self._incorporate_permutation(_gamma, n)
> 208 sage_free(_gamma)
>
> Because sage_malloc is #defined to malloc in stdsage.h,
> I think there
> is a significant potential for memory leaks in code like this. Are we
> thinking correctly on this issue?
Yes. In the above code one could easily cause a serious memory
leak by input a gamma so that gamma[k], for some k with 0 <= k < n,
results in an exception.
> Isn't this a huge problem?
Yes it's a problem.
> Lisandro Dalcin (author of mpi4py) came up with the following trick
> that, while more complicated, prevents memory leaks:
>
> cdef extern from "Python.h":
> object PyString_FromStringAndSize(char*,Py_ssize_t)
> char* PyString_AS_STRING(object)
>
> cdef inline object pyalloc_i(int size, int **i):
> if size < 0: size = 0
> cdef Py_ssize_t n = size * sizeof(int)
> cdef object ob = PyString_FromStringAndSize(NULL, n)
> i[0] = <int*> PyString_AS_STRING(ob)
> return ob
>
> and now
>
> def foo(sequence):
> cdef int size = len(sequence),
> cdef int *buf = NULL
> cdef object tmp = pyalloc_i(size, &buf)
>
> This could probably be adapted into a malloc-like function. What do
> people think?
Could you explain what the point is? Is it that this is a trick so that
Cython will correctly garbage collect the allocated memory, even
if an exception occurs?
-- William
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
