[Numpy-discussion] Owndata flag
How can one arbitrarily assumes that an ndarray owns its data ? More explicitly, I have some temporary home-made C structure that holds a pointer to an array. I prepare (using Cython) an numpy.ndarray using the PyArray_NewFromDescr function. I can delete my temporary C structure without freeing the memory holding array, but I wish the numpy.ndarray becomes the owner of the data. How can do I do such thing ? -- Fabrice Silva ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Owndata flag
On Thu, Dec 15, 2011 at 16:17, Fabrice Silva si...@lma.cnrs-mrs.fr wrote: How can one arbitrarily assumes that an ndarray owns its data ? More explicitly, I have some temporary home-made C structure that holds a pointer to an array. I prepare (using Cython) an numpy.ndarray using the PyArray_NewFromDescr function. I can delete my temporary C structure without freeing the memory holding array, but I wish the numpy.ndarray becomes the owner of the data. How can do I do such thing ? You can't, really. numpy-owned arrays will be deallocated with numpy's deallocator. This may not be the appropriate deallocator for memory that your library allocated. If at all possible, I recommend using numpy to create the ndarray and pass that pointer to your library. Sometimes the library's API gets in the way of this. Otherwise, copy the data. Devs, looking into this, I noticed that we use PyDataMem_NEW() and PyDataMem_FREE() (which is #defined to malloc() and free()) for handling the data pointer. Why aren't we using the appropriate PyMem_*() functions (or the PyArray_*() memory functions which default to using the PyMem_*() implementations)? Using the PyMem_*() functions lets the Python memory manager have an accurate idea how much memory is being used, which can be important for the large amounts of memory that numpy arrays can consume. I assume this is intentional design. I just want to know the rationale for it and would like it documented. I can certainly understand if it causes bad interactions with the garbage collector, say (though hiding information from the GC seems like a suboptimal approach). -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Ann: OpenOpt and FuncDesigner 0.37
Hi all, I'm glad to inform you about new release 0.37 (2011-Dec-15) of our free software: OpenOpt (numerical optimization): IPOPT initialization time gap (time till first iteration) for FuncDesigner models has been decreased Some improvements and bugfixes for interalg, especially for search all SNLE solutions mode (Systems of Non Linear Equations) Eigenvalue problems (EIG) (in both OpenOpt and FuncDesigner) Equality constraints for GLP (global) solver de Some changes for goldenSection ftol stop criterion GUI func manage - now button Enough works in Python3, but Run/Pause not yet (probably something with threading and it will be fixed in Python instead) FuncDesigner: Major sparse Automatic differentiation improvements for badly-vectorized or unvectorized problems with lots of constraints (except of box bounds); some problems now work many times or orders faster (of course not faster than vectorized problems with insufficient number of variable arrays). It is recommended to retest your large-scale problems with useSparse = 'auto' | True| False Two new methods for splines to check their quality: plot and residual Solving ODE dy/dt = f(t) with specifiable accuracy by interalg Speedup for solving 1-dimensional IP by interalg SpaceFuncs and DerApproximator: Some code cleanup You may trace OpenOpt development information in our recently created entries in Twitter and Facebook, see http://openopt.org for details. See also: FuturePlans, this release announcement in OpenOpt forum Regards, D. ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Owndata flag
Am 15.12.2011 um 17:17 schrieb Fabrice Silva: How can one arbitrarily assumes that an ndarray owns its data ? More explicitly, I have some temporary home-made C structure that holds a pointer to an array. I prepare (using Cython) an numpy.ndarray using the PyArray_NewFromDescr function. I can delete my temporary C structure without freeing the memory holding array, but I wish the numpy.ndarray becomes the owner of the data. How can do I do such thing ? There is an excellent blog entry from Travis Oliphant, that describes how to create a ndarray from existing data without copy: http://blog.enthought.com/?p=62 The created array does not actually own the data, but its base attribute points to an object, which frees the memory if the numpy array gets deallocated. I guess this is the behavior you want to achieve. Here is a cython implementation (for a uint8 array) Gregor see 'NumPy arrays with pre-allocated memory', http://blog.enthought.com/?p=62 import numpy as np from numpy cimport import_array, ndarray, npy_intp, set_array_base, PyArray_SimpleNewFromData, NPY_DOUBLE, NPY_INT, NPY_UINT8 cdef extern from stdlib.h: void* malloc(int size) void free(void *ptr) cdef class MemoryReleaser: cdef void* memory def __cinit__(self): self.memory = NULL def __dealloc__(self): if self.memory: #release memory free(self.memory) print memory released, hex(longself.memory) cdef MemoryReleaser MemoryReleaserFactory(void* ptr): cdef MemoryReleaser mr = MemoryReleaser.__new__(MemoryReleaser) mr.memory = ptr return mr cdef ndarray frompointer(void* ptr, int nbytes): import_array() #cdef int dims[1] #dims[0] = nbytes cdef npy_intp dims = npy_intpnbytes cdef ndarray arr = PyArray_SimpleNewFromData(1, dims, NPY_UINT8, ptr) #TODO: check for error set_array_base(arr, MemoryReleaserFactory(ptr)) return arr def test_new_array_from_pointer(): nbytes = 16 cdef void* mem = malloc(nbytes) print memory allocated, hex(longmem) return frompointer(mem, nbytes) ___ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
