Re: [Numpy-discussion] C++ class encapsulating ctypes-numpy array?
Hi Joris Also take a look at the work done by Neal Becker, and posted on this list earlier this year or end of last. Please go ahead and create a cookbook entry on the wiki -- that way we have a central plce for writing up further explorations of this kind (also, let us know on the list if you do). Thanks! Stéfan On Thu, Mar 20, 2008 at 1:46 PM, Joris De Ridder [EMAIL PROTECTED] wrote: Thanks Matthieu, for the interesting pointer. My goal was to be able to use ctypes, though, to avoid having to do manual memory management. Meanwhile, I was able to code something in C++ that may be useful (see attachment). It (should) work as follows. 1) On the Python side: convert a numpy array to a ctypes-structure, and feed this to the C-function: arg = c_ndarray(array) mylib.myfunc(arg) 2) On the C++ side: receive the numpy array in a C-structure: myfunc(numpyArraydouble array) 3) Again on the C++ side: convert the C-structure to an Ndarray class: (e.g. for a 3D array) Ndarraydouble,3 a(array) No data copying is involved in any conversion, of course. Step 2 is required to keep ctypes happy. I can now use a[i][j][k] and the conversion from [i][j][k] to i*strides[0] + j * strides[1] + k * strides[2] is done at compile time using template metaprogramming. The price to pay is that the number of dimensions of the Ndarray has to be known at compile time (to instantiate the template), which is reasonable I think, for the gain in convenience. My first tests seem to be satisfying. I would really appreciate if someone could have a look at it and tell me if it can be done much better than what I cooked. If it turns out that it may interest more people, I'll put it on the scipy wiki. Cheers, Joris ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C++ class encapsulating ctypes-numpy array?
Thanks Matthieu, for the interesting pointer.My goal was to be able to use ctypes, though, to avoid having to do manual memory management. Meanwhile, I was able to code something in C++ that may be useful (see attachment). It (should) work as follows.1) On the Python side: convert a numpy array to a ctypes-structure, and feed this to the C-function: arg = c_ndarray(array) mylib.myfunc(arg)2) On the C++ side: receive the numpy array in a C-structure: myfunc(numpyArraydouble> array) 3) Again on the C++ side: convert the C-structure to an Ndarray class: (e.g. for a 3D array) Ndarraydouble,3> a(array)No data copying is involved in any conversion, of course. Step 2 is required to keep ctypes happy. I can now use a[i][j][k] and the conversion from [i][j][k] to i*strides[0] + j * strides[1] + k * strides[2] is done at compile time using template metaprogramming. The price to pay is that the number of dimensions of the Ndarray has to be known at compile time (to instantiate the template), which is reasonable I think, for the gain in convenience. My first tests seem to be satisfying.I would really appreciate if someone could have a look at it and tell me if it can be done much better than what I cooked.If it turns out that it may interest more people, I'll put it on the scipy wiki.Cheers,Joris ndarray.h Description: Binary data On 19 Mar 2008, at 16:22, Matthieu Brucher wrote:Hi,On my blog, I spoke about the class we used. It is not derived from a Numpy array, it is implemented in terms of a Numpy array (http://matt.eifelle.com/item/5) Matthieu2008/3/19, Joris De Ridder [EMAIL PROTECTED]>: Hi, I'm passing (possibly non-contiguous) numpy arrays (data + shape + strides + ndim) with ctypes to my C++ function (with external "C" to make ctypes happy). Has anyone made a C++ class derived from a ctypes- numpy-array with an overloaded [] operator to allow easy indexing (e.g. x[0][2][5] for a 3D array) so that you don't have to worry about strides? I guess I'm not the first one thinking about this... Cheers, JorisDisclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion -- French PhD studentWebsite : http://matthieu-brucher.developpez.com/Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher ___Numpy-discussion mailing listNumpy-discussion@scipy.orghttp://projects.scipy.org/mailman/listinfo/numpy-discussion___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C++ class encapsulating ctypes-numpy array?
2008/3/20, Joris De Ridder [EMAIL PROTECTED]: Thanks Matthieu, for the interesting pointer. My goal was to be able to use ctypes, though, to avoid having to do manual memory management. Meanwhile, I was able to code something in C++ that may be useful (see attachment). It (should) work as follows. 1) On the Python side: convert a numpy array to a ctypes-structure, and feed this to the C-function: arg = c_ndarray(array) mylib.myfunc(arg) 2) On the C++ side: receive the numpy array in a C-structure: myfunc(numpyArraydouble array) 3) Again on the C++ side: convert the C-structure to an Ndarray class: ( e.g. for a 3D array) Ndarraydouble,3 a(array) No data copying is involved in any conversion, of course. Step 2 is required to keep ctypes happy. I can now use a[i][j][k] and the conversion from [i][j][k] to i*strides[0] + j * strides[1] + k * strides[2] is done at compile time using template metaprogramming. The price to pay is that the number of dimensions of the Ndarray has to be known at compile time (to instantiate the template), which is reasonable I think, for the gain in convenience. My first tests seem to be satisfying. I would really appreciate if someone could have a look at it and tell me if it can be done much better than what I cooked. If it turns out that it may interest more people, I'll put it on the scipy wiki. Cheers, Joris You can use ctypes if and ony if the C++ object is only used in one function call. You can't for instance create a C++ container with ctypes, then in Python call some method and then delete the container, because ctypes will destroy the data after the C++ container was built. This is the only drawback of ctypes. When it comes to strides, you have to divide them by the size of your data : the stride is counted in bytes and not in short/float/... Matthieu -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C++ class encapsulating ctypes-numpy array?
2008/3/20, Joris De Ridder [EMAIL PROTECTED]:
You can use ctypes if and ony if the C++ object is only used in one
function call. You can't for instance create a C++ container with
ctypes, then in Python call some method and then delete the
container, because ctypes will destroy the data after the C++
container was built. This is the only drawback of ctypes.
I'm not sure I understand. Could you perhaps give a pointer for
additional info, or an example?
Suppose you have a C++ class :
struct MyClass
{
MyClass(float* data, int dim, ...)
:container(data, dim)
void method()
{
// Modify container
}
private:
MyContainer container;
};
If the MyContainer class wraps the data array without copying it, if in
Python, you wrap it like :
class MyClass:
def __init__(self, data):
self._inst = #use a C bridge to create a new MyClass from data
def method(self):
wrapMethod(self._inst) #wrapper around method from MyClass
after you create a new Python MyClass, your actual data inside the C++ class
will be freed and thus you have reads or writes errors (and thus can lead to
segmentation faults).
When it comes to strides, you have to divide them by the size of
your data : the stride is counted in bytes and not in short/float/...
Yep, I did this on the Python side. Thanks for the remark, though.
OK ;)
Matthieu
--
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] C++ class encapsulating ctypes-numpy array?
Hi, I'm passing (possibly non-contiguous) numpy arrays (data + shape + strides + ndim) with ctypes to my C++ function (with external C to make ctypes happy). Has anyone made a C++ class derived from a ctypes- numpy-array with an overloaded [] operator to allow easy indexing (e.g. x[0][2][5] for a 3D array) so that you don't have to worry about strides? I guess I'm not the first one thinking about this... Cheers, Joris Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] C++ class encapsulating ctypes-numpy array?
Hi, On my blog, I spoke about the class we used. It is not derived from a Numpy array, it is implemented in terms of a Numpy array ( http://matt.eifelle.com/item/5) Matthieu 2008/3/19, Joris De Ridder [EMAIL PROTECTED]: Hi, I'm passing (possibly non-contiguous) numpy arrays (data + shape + strides + ndim) with ctypes to my C++ function (with external C to make ctypes happy). Has anyone made a C++ class derived from a ctypes- numpy-array with an overloaded [] operator to allow easy indexing (e.g. x[0][2][5] for a 3D array) so that you don't have to worry about strides? I guess I'm not the first one thinking about this... Cheers, Joris Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion -- French PhD student Website : http://matthieu-brucher.developpez.com/ Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn : http://www.linkedin.com/in/matthieubrucher ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
