Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-12 Thread Christopher Barker
Alexander Schmolck wrote:
  I just saw a closely related question posted one
 week ago here (albeit mostly from a swig context).

SWIG, Boost, whatever, the issues are similar. I guess what I'd love to 
find is an array implementation that plays well with modern C++, and 
also numpy.


  The code currently mostly just uses
 plain C double arrays passed around by pointers and I'd like to encapsulate
 this at least with something like stl::vector (or maybe valarray), but I've
 been wondering whether it might not make sense to use (slightly wrapped) numpy
 ndarrays --

Well, you can go back and forth between pointers to data blacks and 
numpy arrays pretty easily. Where you thinking of doing this at the 
python-C++ interface, or where you looking for something you could use 
throughout your code. If the later, then I expect you don't want to use 
a Python Object (unless you're using your code only from Python).

Our case is such: We want to have a nice array-like container that we 
can use in C++ code that makes sense both for pure C++, and interacts 
well with numpy arrays, as the code may be used in pure C++ app, but 
also want to test it, script it, etc from Python.

 Also, ndarrays
 provide fairly rich functionality even at the C-API-level

Yes, the more I look into this, the more I'm impressed with numpy's design.


 but there doesn't seem to be one obvious choice, as
 there is for python. 

Though there may be more than one good choice -- did you check out 
boost::multiarray ? I didn't see that on your list.

 Things that would eventually come in handy, although they're not needed yet,
 are basic linear algebra and maybe two or three LAPACK-level functions (I can
 think of cholesky decomposition and SVD right now)

It would be nice to just have that (is MTL viable?), but writing 
connection code to LAPACK for a few functions is not too bad.

 I think I could get all these things (and more) from scipy
 (and kin) with too much fuzz (although I haven't tried wavelet support yet)
 and it seems like picking together the same functionality from different C++
 libs would require considerably more work.

True -- do-able, but you'd have to do it!

 So my question is: might it make sense to use (a slightly wrapped)
 numpy.ndarray,

I guess what I'd like is a C++ array that was essentially an ndarray 
without the pyobject stuff -- it could then be useful for C++, but also 
easy to go back and forth between numpy and C++.

Ideally, there'd be something that already fits that bill. I see a 
couple design issues that are key:

View semantics: numpy arrays have the idea of views of data built in 
to them -- a given array can have it's own data block, or a be a view 
onto another. This is quite powerful and flexible, and can save a lot a 
data copying. The STL containers don't seem to have that concept at all. 
std::valarray has utility classes that are views of a valarray, but they 
really only useful as temporaries - they are not full-blown valarrays.

It looks like boost::multiarrays have a similar concept though

The MultiArray concept defines an interface to hierarchically nested 
containers. It specifies operations for accessing elements, traversing 
containers, and creating views of array data.


Another issue is dynamic typing. Templates provide a way to do generic 
programming, but it's only generic at the code level. At compile time, 
types are fixed, so you have a valarraydouble, for instance. numpy 
arrays, on the other hand are of only one type - with the data type 
specified as meta-data essentially. I don't know what mismatch this may 
cause, but it's a pretty different way to structure things. (Side note: 
I used this feature once to re-type an array in place, using the same 
data block -- it was a nifty hack used to unpack an odd binary format). 
Would it make sense to use this approach in C++? I suspect not -- all 
your computational code would have to deal with it.

There is also the re-sizing issue. It's pretty handy to be able to 
re-size arrays -- but then the data pointer can change, making it pretty 
impossible to share the data. Maybe it would be helpful to have a 
pointer-to-a-pointer instead, so that the shared pointer wouldn't 
change. However, there could be uglyness with the pointer changing while 
some other view is working with it.

 http://thread.gmane.org/gmane.comp.python.c++/11559/focus=11560

That does look promising -- and it used boost::multiarrays

The more I look at boost::multiarray, the better I like it (and the more 
it looks like numpy) -- does anyone here have experience (good or bad) 
with it?

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

NOAA/ORR/HAZMAT (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115   (206) 526-6317   main reception
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-12 Thread David Cournapeau
Christopher Barker wrote:
 Alexander Schmolck wrote:
  I just saw a closely related question posted one
 week ago here (albeit mostly from a swig context).

 SWIG, Boost, whatever, the issues are similar. I guess what I'd love to 
 find is an array implementation that plays well with modern C++, and 
 also numpy.

Maybe I am naive, but I think a worthy goal would be a minimal C++ 
library which wraps ndarray, without thinking about SWIG, boost and co 
first. I don't know what other people are looking for, but for me, the 
interesting things with using C++ for ndarrays would be (in this order 
of importance):
1 much less error prone memory management
2 a bit more high level than plain C ndarrays (syntactic sugar 
mostly: keyword args, overloaded methods and so on)
3 more high level things for views
4 actual computation (linear algebra, SVD, etc...)

1 and 2 can easily be implemented without anything but plain C++. 3 
would be a pain to do right without e.g boost::multiarray; 4 obviously 
needs external libraries anyway. My guess is that most people need 
mostly those 4 points, but not in the same order. Am I right ?

One huge advantage of being independant of external libraries would be 
that the wrapper could then be included in numpy, and you could expect 
it everywhere.

 I guess what I'd like is a C++ array that was essentially an ndarray 
 without the pyobject stuff -- it could then be useful for C++, but also 
 easy to go back and forth between numpy and C++.

 Ideally, there'd be something that already fits that bill. I see a 
 couple design issues that are key:

 View semantics: numpy arrays have the idea of views of data built in 
 to them -- a given array can have it's own data block, or a be a view 
 onto another. This is quite powerful and flexible, and can save a lot a 
 data copying. The STL containers don't seem to have that concept at all. 
 std::valarray has utility classes that are views of a valarray, but they 
 really only useful as temporaries - they are not full-blown valarrays.
What do you mean by STL does not have the concept of view ? Do you mean 
vector ?

 It looks like boost::multiarrays have a similar concept though
 
 The MultiArray concept defines an interface to hierarchically nested 
 containers. It specifies operations for accessing elements, traversing 
 containers, and creating views of array data.
 

 Another issue is dynamic typing. Templates provide a way to do generic 
 programming, but it's only generic at the code level. At compile time, 
 types are fixed, so you have a valarraydouble, for instance. numpy 
 arrays, on the other hand are of only one type - with the data type 
 specified as meta-data essentially. I don't know what mismatch this may 
 cause, but it's a pretty different way to structure things. (Side note: 
 I used this feature once to re-type an array in place, using the same 
 data block -- it was a nifty hack used to unpack an odd binary format). 
 Would it make sense to use this approach in C++? I suspect not -- all 
 your computational code would have to deal with it.
Why not making one non template class, and having all the work done 
inside the class instead ?

class ndarray {
private:
 ndarray_impdouble a;
};


 There is also the re-sizing issue. It's pretty handy to be able to 
 re-size arrays -- but then the data pointer can change, making it pretty 
 impossible to share the data. Maybe it would be helpful to have a 
 pointer-to-a-pointer instead, so that the shared pointer wouldn't 
 change. However, there could be uglyness with the pointer changing while 
 some other view is working with it.
If you have an array with several views on it, why not just enforcing 
that the block data address cannot change as long as you have a view ? 
This should not be too complicated, right ? I don't use views that much 
myself in numpy (other than implicetely, of course), so I may missing 
something important here

cheers,

David

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-12 Thread Alexander Schmolck
Christopher Barker [EMAIL PROTECTED] writes:

 Alexander Schmolck wrote:
  I just saw a closely related question posted one
 week ago here (albeit mostly from a swig context).

 SWIG, Boost, whatever, the issues are similar. I guess what I'd love to 
 find is an array implementation that plays well with modern C++, and 
 also numpy.


 The code currently mostly just uses plain C double arrays passed around by
 pointers and I'd like to encapsulate this at least with something like
 stl::vector (or maybe valarray), but I've been wondering whether it might
 not make sense to use (slightly wrapped) numpy ndarrays --

 Well, you can go back and forth between pointers to data blacks and 
 numpy arrays pretty easily. Where you thinking of doing this at the 
 python-C++ interface, or where you looking for something you could use 
 throughout your code. 

The latter -- I'd ideally like something that I can more or less transparently
pass and return data between python and C++ and I want to use numpy arrays on
the python side. It'd also be nice to have reference semantics and reference
counting working fairly painlessly between both sides.

 If the later, then I expect you don't want to use a Python Object (unless
 you're using your code only from Python).

Yup; that would be somewhat perverse -- although as I said I expect that most
data I deal with will be pretty large, so overheads from creating python
objects aren't likely to matter that much.

 Our case is such: We want to have a nice array-like container that we 
 can use in C++ code that makes sense both for pure C++, and interacts 
 well with numpy arrays, as the code may be used in pure C++ app, but 
 also want to test it, script it, etc from Python.

Yes, that's exactly what I'm after. What's your current solution for this?

 Also, ndarrays
 provide fairly rich functionality even at the C-API-level

 Yes, the more I look into this, the more I'm impressed with numpy's design.


 but there doesn't seem to be one obvious choice, as
 there is for python. 

 Though there may be more than one good choice -- did you check out 
 boost::multiarray ? I didn't see that on your list.

No, I hadn't looked at that -- thanks. It looks like a raw, stripped down
version of a multidimensional array -- no . Since I'm mostly going to use
matrices (and vectors, here and there), maybe ublas, which does provide useful
numeric functionality is a better choice. I must say I find it fairly painful
to figure out how to do things I consider quite basic with the matrix/array
classes I come accross in C++ (I'm not exactly a C++ expert, but still); I
also can't seem to find a way to construct an ublas matrix or vector from
existing C-array data. 

 Things that would eventually come in handy, although they're not needed yet,
 are basic linear algebra and maybe two or three LAPACK-level functions (I can
 think of cholesky decomposition and SVD right now)

 It would be nice to just have that (is MTL viable?)

No idea -- as far as I can tell the webpage is broken, so I can't look at the
examples (http://osl.iu.edu/research/mtl/examples.php3). It doesn't seem to
provide SVD out of th box either though -- and since I've already got a boost
dependency my first instinct would be to use something from there. What's the
advantage of MTL over ublas?

 but writing connection code to LAPACK for a few functions is not too bad.


 I think I could get all these things (and more) from scipy
 (and kin) with too much fuzz (although I haven't tried wavelet support yet)
 and it seems like picking together the same functionality from different C++
 libs would require considerably more work.

 True -- do-able, but you'd have to do it!

 So my question is: might it make sense to use (a slightly wrapped)
 numpy.ndarray,

 I guess what I'd like is a C++ array that was essentially an ndarray 
 without the pyobject stuff -- it could then be useful for C++, but also 
 easy to go back and forth between numpy and C++.

Indeed.

 Ideally, there'd be something that already fits that bill. I see a 
 couple design issues that are key:

 View semantics: numpy arrays have the idea of views of data built in 
 to them -- a given array can have it's own data block, or a be a view 
 onto another. This is quite powerful and flexible, and can save a lot a 
 data copying. The STL containers don't seem to have that concept at all. 

Yes. C++ copying semantics seem completely braindamaged to me.

 std::valarray has utility classes that are views of a valarray, but they 
 really only useful as temporaries - they are not full-blown valarrays.

 It looks like boost::multiarrays have a similar concept though
 
 The MultiArray concept defines an interface to hierarchically nested 
 containers. It specifies operations for accessing elements, traversing 
 containers, and creating views of array data.
 

 Another issue is dynamic typing. Templates provide a way to do generic 
 programming, but it's only generic at the code level. At 

Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-12 Thread Christopher Barker
David Cournapeau wrote:
 Maybe I am naive, but I think a worthy goal would be a minimal C++ 
 library which wraps ndarray, without thinking about SWIG, boost and co 
 first.

That's exactly what I had in mind. If you have something that works well 
with ndarray -- then SWIG et al. can work with it. In principle, if you 
can do the transition nicely with hand-written wrappers, then you can do 
it with the automated tools too.

  I don't know what other people are looking for, but for me, the
 interesting things with using C++ for ndarrays would be (in this order 
 of importance):
 1 much less error prone memory management
less than what? std:valarray, etc. all help with this.

 2 a bit more high level than plain C ndarrays (syntactic sugar 
 mostly: keyword args, overloaded methods and so on)
Yes.
 3 more high level things for views
I think views are key.
 4 actual computation (linear algebra, SVD, etc...)

This is last on my list -- key is the core data type. I may be an 
unusual user, but what I expect is that in a given pile of code, we need 
one or two linear algebra routines, so I don't mind hand-wrapping 
LAPACK. Not that it wouldn't be nice to have it built in, but it's not a 
deal breaker. In any case, it should be separate: a core set of array 
objects, an a linear algebra (or whatever else) package built on top of it.

 3 
 would be a pain to do right without e.g boost::multiarray;

Yes, it sure would be nice to build it on an existing code base, and 
boost::multiarray seems to fit.

 One huge advantage of being independant of external libraries would be 
 that the wrapper could then be included in numpy, and you could expect 
 it everywhere.

That would be nice, but may be too much work.

Im really a C++ newbie, but it seems like the key here is the view 
semantics -- and perhaps the core solution is to have a data block 
class -- all it would have is a pointer to a block of data, and a 
reference counter. Then each array object would have a view of one of 
those -- each new array object that used a given instance would increase 
the ref count, and decrease it on deletion. The view would destroy 
itself when its refcount went to zero. (is this how numpy works now?)

Even if this makes sense, I have no idea how compatible it would be with 
numpy and/or python.

boost:multiarray does not seem to take this approach. Rather it has two 
classes: a multi_array: responsible for its own data block, and a 
multi_array_ref: which uses a view on another multiarray's data block. 
This is getting close, but it means that when you create a 
multi_array_ref, the original multi_array needs to stay around. I'd 
rather have much more flexible system,where you could create an array, 
create a view of that array, then destroy the original, then have the 
data block go away when you destroy the view. This could cause little 
complications if you started with a huge array, made a view into a tiny 
piece of it, then the whole data block would stick around -- but that 
would be up to the user to think about.

 Would it make sense to use this approach in C++? I suspect not -- all 
 your computational code would have to deal with it.
 Why not making one non template class, and having all the work done 
 inside the class instead ?
 
 class ndarray {
 private:
  ndarray_impdouble a;
 };

hm. that could work (as far as my limited C++ knowledge tells me),b ut 
it's still static at run time -- which may be OK -- and is C++-is anyway.

 If you have an array with several views on it, why not just enforcing 
 that the block data address cannot change as long as you have a view ? 

Maybe Im missing what you're suggesting but this would lock in the 
original array once any views were on it -- that would greatly restrict 
flexibility. My suggestion above may help, but I think maybe I could 
just live without re-sizing.

 This should not be too complicated, right ? I don't use views that much 
 myself in numpy (other than implicitly, of course), so I may missing 
 something important here

Implicitly, we're all using them all the time -- which is why I think 
views are key.

Alexander Schmolck wrote:
 I'd ideally like something that I can more or less transparently
 pass and return data between python and C++ and I want to use numpy arrays on
 the python side. It'd also be nice to have reference semantics and reference
 counting working fairly painlessly between both sides.

Can the python gurus here comment on how possible that is?

 as I said I expect that most
 data I deal with will be pretty large, so overheads from creating python
 objects aren't likely to matter that much.

Im not so much worried about the overhead as the dependency -- to use 
your words, it would feel perverse to by including python.h for a 
program that wasn't using python at all.

 Our case is such: We want to have a nice array-like container that we 
 can use in C++ code that makes sense both for pure C++, and interacts 
 well with 

Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-12 Thread Matthieu Brucher

 less than what? std:valarray, etc. all help with this.



I do not agree with this statement. A correct memory managed array would
increment and decrement a reference counter somewhere.


Yes, it sure would be nice to build it on an existing code base, and
 boost::multiarray seems to fit.



The problem with multiarray is that the dimension of the array are fixed at
the compilation. Although one could use 1 for the size in the remaning
dimension, I don't think it's the best choice, given that a real
dynamic-dimension array is not much complicated than a static-dimension one,
perhaps a little more slower.


boost:multiarray does not seem to take this approach. Rather it has two
 classes: a multi_array: responsible for its own data block, and a
 multi_array_ref: which uses a view on another multiarray's data block.
 This is getting close, but it means that when you create a
 multi_array_ref, the original multi_array needs to stay around. I'd
 rather have much more flexible system,where you could create an array,
 create a view of that array, then destroy the original, then have the
 data block go away when you destroy the view. This could cause little
 complications if you started with a huge array, made a view into a tiny
 piece of it, then the whole data block would stick around -- but that
 would be up to the user to think about.



I don't know ho numpy does it either, but a view on an view of an array may
be a view on an array, so in C++, an view should only reference the data,
not the real view, so when the array is destroyed, the view is still
correct, as it has a reference on the data and not the original array.


hm. that could work (as far as my limited C++ knowledge tells me),b ut
 it's still static at run time -- which may be OK -- and is C++-is anyway.



I've done this before, with type traits and a multi-dispatch method, you can
instantiate several functions with the correct type. It's a classic approach
that is used in plugins, and it does not use RTTI, and it is compatible
across C++ compilers.


Can the python gurus here comment on how possible that is?



Once you have the Python object, increment the reference counter when you
wrap the data in C++ for a real array or for a view, and decrement it in the
destructor of your C++ object, is that what you mean ?
If the C++ object can directly use a PyObject, it's very simple to use. It
perhaps could be done by a policy class, so that temporary C++ object would
use a default policy that does not rely on a Python object.


Im not so much worried about the overhead as the dependency -- to use
 your words, it would feel perverse to by including python.h for a
 program that wasn't using python at all.



This is solved if one can use policy classes.


This functionality seems to be missing from many (moat) of these C++
 containers. I suspect that it's the memory management issue. One of the
 points of these containers it to take care of memory management for you
 -- if you pass in a pointer to an existing data block -- it's not
 managing your memory any more.



What Albert did for hs wrapper is this : provide an adaptator that can use
the data pointer. It's only a policy (but not the default one).


Full Disclosure: I have neither the skills nor the time to actually
 implement any of these ideas. If no one else does, then I guess we're
 just blabbing -- not that there is anything wrong with blabbing!


I know that in my lab, we intend to wrap numpy arrays in a C++ multi array,
but not the boost one. It will be for array that have more than 3
dimensions, and for less than 2 dimensions, we will use our own matrix
library, as it is simple to wrap array with it. The most complicated thing
will be the automatic conversion. It will most likely be Open Source (GPL),
but I don't know when we will be able to have time to do it and then when we
will make it available...

Matthieu
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-12 Thread Neal Becker
Travis E. Oliphant wrote:

 
 nd to copy hundres of MB around unnecessarily.

 I think it is a real shame that boost currently doesn't properly support
 numpy out of the box, although numpy has long obsoleted both numarray and
 Numeric (which is both buggy and completely unsupported). All the more so
 since writing multimedial or scientific extensions (in which numpy's
 array interface is very natural to figure prominently) would seem such an
 ideal use for boost.python, as soon as complex classes or compound
 structures that need to efficiently support several (primitive) datatypes
 are involved, boost.python could really play its strenghts compared to
 Fortran/C based extensions.

   
 I think it could be that boost.python is waiting for the extended buffer
 interface which is coming in Python 3.0 and Python 2.6.  This would
 really be ideal for wrapping external code in a form that plays well
 with other libraries.
 
 
 -Travis O.

I've spent a lot of time on this issue as well.  There have been a few
efforts, and at present I've followed my own path.

My interest is in exposing algorithms that are written in generic c++ style
to python.

The requirement here is to find containers (vectors, n-dim arrays) that are
friendly to the generic c++ side and can be used from python.  My opinion
is that Numeric and all it's descendants aren't what I want on the c++
interface side.  Also, I've stumbled over trying to grok ndarrayobject.h,
and I haven't had much success finding docs.

What I've done instead is to basically write all that I need from numpy
myself.  I've usually used ublas vector and matrix to do this.  I've also
used boost::multi_array at times (and found it quite good), and fixed_array
from stlsoft.  I implemented all the arithmetic I need and many functions
that operate on vectors (mostly I'm interested in vectors to represent
signals - not so much higher dimen arrays).

As far as views of arrays, ref counting, etc.  I have not worried much about
it.  I thought it would be a very elegant idea, but in practice I don't
really need it.  The most common thing I'd do with a view is to operate on
a slice.  Python supports this via __setitem__.  For example:
u[4:10:3] += 2
works.  There is no need for python to hold a reference to a vector slice to
do this.

Probably the biggest problem I've encountered is that there is not any
perfect c++ array container.  For 1-dimen, std::vector is pretty good - and
the interface is reasonable.  For more dimen, there doesn't seem to be any
perfect solution or general agreement on interface (or semantics).

One of my favorite ideas related to this.  I've gotten a lot of mileage out
of moving from the pair-of-iterator interface featured by stl to the
boost::range.  I believe it would be useful to consider a multi-dimen
extension of this idea.  Perhaps this could present some unifying interface
to different underlying array libraries.  For example, maybe something
like:

templatetypename in_t
void F (in_t const in) {
  typename row_iteratorin_t::type r = row_begin (in);
  for (; r != row_end (in); ++r) {
typename col_iteratorin_t::type c = col_begin (r);
...

The idea is that even though multi_array and ublas::matrix present very
different interfaces, they can be adapted to a 2-dimen range abstraction. 
Anyway, that's a different issue.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


[Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-11 Thread Alexander Schmolck
Hi,

I've sent pretty much the same email to c++sig, but I thought I'd also try my
luck here, especially since I just saw a closely related question posted one
week ago here (albeit mostly from a swig context).

I'm working working on an existing scientific code base that's mostly C++ and
I'm currently interfacing it to python with boost.python with a view to doing
non-performance critical things in python. The code currently mostly just uses
plain C double arrays passed around by pointers and I'd like to encapsulate
this at least with something like stl::vector (or maybe valarray), but I've
been wondering whether it might not make sense to use (slightly wrapped) numpy
ndarrays -- since I eventually plan to make fairly heavy use of existing
python infrastructure like matplotlib and scipy where possible. Also, ndarrays
provide fairly rich functionality even at the C-API-level and I'm pretty
familiar with numpy. Furthermore I can't find something equivalent to numpy
for C++ -- there's ublas as well as several other matrix libs and a couple of
array ones (like blitz++), but there doesn't seem to be one obvious choice, as
there is for python. I think I will mostly use double arrays of fairly large
size, so having really low overhead operations on small arrays with more or
less exotic types is not important to me.

Things that would eventually come in handy, although they're not needed yet,
are basic linear algebra and maybe two or three LAPACK-level functions (I can
think of cholesky decomposition and SVD right now) as well as possibly
wavelets (DWT). I think I could get all these things (and more) from scipy
(and kin) with too much fuzz (although I haven't tried wavelet support yet)
and it seems like picking together the same functionality from different C++
libs would require considerably more work.

So my question is: might it make sense to use (a slightly wrapped)
numpy.ndarray, and if so is some code already floating around for that (on
first glance it seems like there's a bit of support for the obsolete Numeric
package in boost, but none for the newer numpy that supercedes it); if not is
my impression correct that making the existing code numpy compatible shouldn't
be too much work.

Provided this route doesn't make much sense, I'd also be curious what people
would recommend doing instead.

In last week's thread mentioned above I found the following link which looks
pretty relevant, albeit essentially undocumented and possibly pre-alpha -- has
anyone here tried it out?

http://thread.gmane.org/gmane.comp.python.c++/11559/focus=11560

many thanks,

'as

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-11 Thread Philip Austin
Alexander Schmolck writes:

  So my question is: might it make sense to use (a slightly wrapped)
  numpy.ndarray, and if so is some code already floating around for that (on
  first glance it seems like there's a bit of support for the obsolete Numeric
  package in boost, but none for the newer numpy that supercedes it); if not is
  my impression correct that making the existing code numpy compatible 
  shouldn't
  be too much work.

Right, it should work more or less as is if you just do:

set_module_and_type(numpy, ArrayType);

in the examples.  Some tests will fail because of numpy changes to
function signatures, etc.

The current library doesn't wrap numpy.zeros, numpy.ones or
numpy.empty constructors, so the only way to construct an empty
is to pass the constructor a tuple and then resize.  Because
it (by design) doesn't include arrayobject.h, there's also no clean
way to get at the underlying data pointer to share the memory.  You
can use helper functions like this, though:

//Create a one-dimensional numpy array of length n and type t
boost::python::numeric::array makeNum(intp n, PyArray_TYPES t=PyArray_DOUBLE){
  object obj(handle(PyArray_FromDims(1, n, t)));
  return extractboost::python::numeric::array(obj);
}

http://www.eos.ubc.ca/research/clouds/software/pythonlibs/num_util/num_util_release2/Readme.html

has more examples -- Phil

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion


Re: [Numpy-discussion] numpy.ndarrays as C++ arrays (wrapped with boost)

2007-09-11 Thread Travis E. Oliphant

 nd to copy hundres of MB around unnecessarily.

 I think it is a real shame that boost currently doesn't properly support numpy
 out of the box, although numpy has long obsoleted both numarray and Numeric
 (which is both buggy and completely unsupported). All the more so since
 writing multimedial or scientific extensions (in which numpy's array interface
 is very natural to figure prominently) would seem such an ideal use for
 boost.python, as soon as complex classes or compound structures that need to
 efficiently support several (primitive) datatypes are involved, boost.python
 could really play its strenghts compared to Fortran/C based extensions.

   
I think it could be that boost.python is waiting for the extended buffer 
interface which is coming in Python 3.0 and Python 2.6.  This would 
really be ideal for wrapping external code in a form that plays well 
with other libraries.


-Travis O.

___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion