Stefan Behnel wrote: > Hi, Hi,
> when re-reading an older thread about the struct syntax, I had a funny idea > I wanted to share. > > Robert Bradshaw wrote: >> I'm with Stefan that it is very >> dangerous to hide the malloc and expect the user to explicitly >> provide the free. If the user wants to manage their own memory, they >> can do so with malloc and friends (understanding the associated >> dangers), and any special memory-management stuff we add should be as >> implicit and easy to use as Python's garbage collection (and probably >> piggyback off of it). > > The "piggybacking" here makes me wonder if it would work to provide a > dedicated "Memory" object, that would be a Python object but could be used > as in > > def do_stuff_with_dynamic_memory(Py_ssize_t size): > cdef Memory mem > cdef void* ptr > mem = Memory(10*size) # equiv to malloc() > ptr = mem # automagic coercion to a pointer to the memory buffer > # do stuff with *ptr, e.g. hand it around in C code > mem = None # => Py_DECREF(mem), equiv to free() > In general I consider this a great idea since people will be less likely to shoot themselves in the foot and as you mentioned for small allocs this should really be a win. Having memory automatically freed on exceptions would also be great since that is tricky to write. The less the average user has to deal with memory allocs and deallocs in Cython code the better Cython will be. But I am concerned to some extend about the success of hunting memory leaks if this is the default allocator in some future version of Cython and there is no way to use malloc() [currenly Sage provide sagemalloc(), so that for example could be made to use do_stuff_with_dynamic_memory() since we only need to change one define]. Looking for heap objects in Python that are leaked or not properly dereferenced is a giant pain right now and valgrind is next to useless here. muppy is starting to solve that problem, but there are at least in Sage issues with ipython allocating tens of thousands of heap objects at load time impacting performance of muppy badly, but those issues will be hopefully fixed. Anyway, no need to rant about python memory heap debugging here :) Since in Sage I have to deal with loads of small, but potentially large cumulative leaks I would consider it very nice if the allocator could have some magic debug fallback to malloc(). That should certainly be not the default behavior, just like --without-pymalloc is not the default for Python. Just my 0.2 cents <SNIP> > Stefan Cheers, Michael > _______________________________________________ > Cython-dev mailing list > [email protected] > http://codespeak.net/mailman/listinfo/cython-dev > _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
