Re: [Numpy-discussion] using numpy array data in C++ matrix template library (MTL) matrices
JJ wrote: Hello: I would like to make a numpy array accessible in C++ as a matrix template library (MTL) matrix object. Has anyone done this before? I see that there is a project numpy_boost that will make numpy arrays accessible as boost::multi_array objects. Any ideas on how I might do this, or even better, a simple example of passing a numpy array to C++, accessing it in a MatrixRow container, changing one element, and then passing it back to python/numpy? I'm a novice C++ programmer, so bear with me. The details will depend on your matrix library, but the underlying numpy array object has a full C api, so you can do whatever you want with it in your C++ code. But it can get quite messy :) I don't know for MTL, and for C++, boost can be useful, like Neal suggested. One thing to keep in mind is that C arrays are much more general that most low level C++ array containers (numpy arrays may not be contiguous in memory, for example). The easy solution is to use the numpy C api to get C contiguous arrays, so that the underlying data pointer can be accessed as expected by a plain C array. At this point, it is no different than using an C pointer pointing to an array, which virtually every C++ array library can deal with. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] using numpy array data in C++ matrix template library (MTL) matrices
David Cournapeau wrote: The details will depend on your matrix library, but the underlying numpy array object has a full C api, so you can do whatever you want with it in your C++ code. But it can get quite messy :) I don't know for MTL, and for C++, boost can be useful, like Neal suggested. One thing to keep in mind is that C arrays are much more general Sorry, this should read keep in mind that *numpy* C array are much more general, cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] array of matrices
On Wed, Apr 01, 2009 at 01:40:54AM +, Hans-Andreas Engel wrote: By the way, matrix multiplication is one of the testcases for the generalized ufuncs in numpy 1.3 -- this makes playing around with it easy: In [1]: N = 10; a = randn(N, 4, 4); b = randn(N, 4, 4) In [2]: import numpy.core.umath_tests In [3]: (numpy.core.umath_tests.matrix_multiply(a, b) == [dot(ai, bi) for (ai, bi) in zip(a, b)]).all() Out[3]: True Hey Hans-Andreas, I am jumping on your message to stress that generalized ufunc realy need a bit more documentation. I have been trying to push a few collegues to use them, but I am getting not traction, because they are unsure of what generalized ufuncs do. I wasn't aware of the example you mentioned. It certainly is useful. Maybe it would be useful to add it as an example to the generalized-ufunc documnetation embryo (http://docs.scipy.org/numpy/docs/numpy-docs/reference/generalized_ufuncs.rst), with a discussion of the example. If you, or anyone else who knows generalized ufuncs, register on the above webpage, I can give rights to edit the page, so that you can improve the docs. Cheers, Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] trouble building docs with sphinx-0.6.1
On Wed, Apr 01, 2009 at 08:40:15PM +, Pauli Virtanen wrote: It was an incompatibility of Numpy's autosummary extension and Sphinx = 0.6. It should now be fixed in Numpy trunk. autosummary is now in Sphinx (= 0.6). Shouldn't we be using Sphinx's version, and default to ours for versions of Sphinx lacking the extention. If we don't, I fear the same problem will arise again in the long run: we have too much tight coupling for a feature that is not essential to numpy. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Got: undefined symbol: PyUnicodeUCS2_FromUnicode error
Hi All, I got the undefined symbol: PyUnicodeUCS2_FromUnicode error when importing numpy. I have my own non-root version of python 2.5.4 final installed with --prefix=$HOME/usr. PYTHONHOME=$HOME/usr; PYTHONPATH=$PYTHONHOME/lib:$PYTHONHOME/lib/python2.5/site-packages/ install and import other modules works fine. I install numpy-1.3.0rc2 from the svn repository with python setup.py install then import numpy results in following error: *Python 2.5 (release25-maint, Jul 23 2008, 17:54:01) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2 Type help, copyright, credits or license for more information. import numpy Traceback (most recent call last): File stdin, line 1, in module File numpy/__init__.py, line 130, in module import add_newdocs File numpy/add_newdocs.py, line 9, in module from lib import add_newdoc File numpy/lib/__init__.py, line 4, in module from type_check import * File numpy/lib/type_check.py, line 8, in module import numpy.core.numeric as _nx File numpy/core/__init__.py, line 5, in module import multiarray ImportError: numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_FromUnicode import Bio import sys* I am not sure where is trick is. As I checked the previous discussion, I found several people raised similar issue but no one has posted a final solution to this yet. So I can only ask for help here again! Thanks in advance! Charlie ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Got: undefined symbol: PyUnicodeUCS2_FromUnicode error
On Wed, Apr 8, 2009 at 03:05, charlie charlie.xia@gmail.com wrote: Hi All, I got the undefined symbol: PyUnicodeUCS2_FromUnicode error when importing numpy. I have my own non-root version of python 2.5.4 final installed with --prefix=$HOME/usr. PYTHONHOME=$HOME/usr; PYTHONPATH=$PYTHONHOME/lib:$PYTHONHOME/lib/python2.5/site-packages/ install and import other modules works fine. I install numpy-1.3.0rc2 from the svn repository with python setup.py install then import numpy results in following error: Are you sure you are using the same python executable to build and run this installation of numpy? This problem is always caused by this. -- Robert Kern I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth. -- Umberto Eco ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Got: undefined symbol: PyUnicodeUCS2_FromUnicode error
charlie wrote: Hi All, I got the undefined symbol: PyUnicodeUCS2_FromUnicode error when importing numpy. I have my own non-root version of python 2.5.4 final installed with --prefix=$HOME/usr. PYTHONHOME=$HOME/usr; PYTHONPATH=$PYTHONHOME/lib:$PYTHONHOME/lib/python2.5/site-packages/ install and import other modules works fine. I install numpy-1.3.0rc2 from the svn repository with python setup.py install then import numpy results in following error: /Python 2.5 (release25-maint, Jul 23 2008, 17:54:01) / As you can see, this is the system -wide python. You built a python which is binary incompatible with the system python - which is almost always a bad idea unless you really know what you are doing. You should use the system wide python, or built a compatible python. On debian, this is done with the option --enable-unicode=ucs4 (from you error message, I guess your installed python was configured with --enable-unicode=ucs2). cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Got: undefined symbol: PyUnicodeUCS2_FromUnicode error
charlie wrote: Hi All, Hi Charlie, I got the undefined symbol: PyUnicodeUCS2_FromUnicode error when importing numpy. I have my own non-root version of python 2.5.4 final installed with --prefix=$HOME/usr. PYTHONHOME=$HOME/usr; PYTHONPATH=$PYTHONHOME/lib:$PYTHONHOME/lib/python2.5/site-packages/ install and import other modules works fine. I install numpy-1.3.0rc2 from the svn repository with python setup.py install then import numpy results in following error: *Python 2.5 (release25-maint, Jul 23 2008, 17:54:01) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2 Type help, copyright, credits or license for more information. SNIP PyUnicodeUCS2_FromUnicode numpy was build with a python configured with ucs2 while the python you have is build with ucs4. import Bio import sys* I am not sure where is trick is. As I checked the previous discussion, I found several people raised similar issue but no one has posted a final solution to this yet. So I can only ask for help here again! Thanks in advance! To fix this build python with ucs2, i.e. check configure --help and set python to use ucs2. Charlie Cheers, Michael ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] trouble building docs with sphinx-0.6.1
Wed, 08 Apr 2009 09:59:50 +0200, Gael Varoquaux kirjoitti: On Wed, Apr 01, 2009 at 08:40:15PM +, Pauli Virtanen wrote: It was an incompatibility of Numpy's autosummary extension and Sphinx = 0.6. It should now be fixed in Numpy trunk. autosummary is now in Sphinx (= 0.6). Shouldn't we be using Sphinx's version, and default to ours for versions of Sphinx lacking the extention. If we don't, I fear the same problem will arise again in the long run: we have too much tight coupling for a feature that is not essential to numpy. The version in Sphinx is incomplete; I've asked Georg to pull fixes from my branch, and after those get merged, we can switch. -- Pauli Virtanen ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Blas / lapack / binary installers...
Hello, Summary: is it possible to distribute, optionally or not, the blas / lapack libraries that numpy is built against, with the numpy binary installers? We at the NIPY project have run into what seems like a recurring problem; we want to build our code against both numpy and lapack, on windows, linux and OS X. No problem of course if we've done a development install - we already needed to have blas/lapack. For a binary install, we've got a nasty problem, especially on windows. Of course, on windows, numpy.distutils dreams that there will be libraries still in the installed location for the builder's machine - I think by default at C:\local\lib\yop\sse3. So, to make - say - NIPY - build, we need to: 1) Download some binary blas/lapack libraries, hoping they match our compiler, for example from links here (for windows): http://www.scipy.org/Installing_SciPy/Windows 2) EITHER: make our own site.cfg specifying the library location and library files, and copy this same site.cfg to any other package we're building against numpy / lapack, OR: edit the site.cfg in the installed package distribution to point to these new libraries. So, I'm wondering, would it be beyond sense to: Proposal - Make the binary installers for windows with the option to put the blas/lapack libraries in some predictable location and adapt the site.cfg file accordingly. Is it possible this would also make compiling scipy on windows a whole lot easier? Thanks a lot, Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Blas / lapack / binary installers...
On Wed, Apr 08, 2009 at 12:44:02PM -0700, Matthew Brett wrote: Summary: is it possible to distribute, optionally or not, the blas / lapack libraries that numpy is built against, with the numpy binary installers? You mean the unlinked libraries (.a or .so), and the corresponding headers, I believe. I would rephrase the problem in a more general way: would it be possible to have so way that a project using scipy or numpy at runtime could use numpy's blas (or even better, scipy's lapack, that way we are garantied to have a real lapack) in its C code. I am not sure at all what the right strategy to do this it, this is why I rephrased Matthew's question in a more general way. Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] dtype field renaming
hi there -- for a numpy.recarray, is it possible to rename the fields in the dtype? thanks a bunch elaine ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] dtype field renaming
On Apr 8, 2009, at 5:57 PM, Elaine Angelino wrote:
hi there --
for a numpy.recarray, is it possible to rename the fields in the
dtype?
Take a new view:
a = np.array([(1,1)],dtype=[('a',int),('b',int)])
b = a.view([(A,int), ('b', int)])
or:
use numpy.lib.recfunctions.rename_fields
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] dtype field renaming
2009/4/9 Pierre GM pgmdevl...@gmail.com:
for a numpy.recarray, is it possible to rename the fields in the
dtype?
Take a new view:
a = np.array([(1,1)],dtype=[('a',int),('b',int)])
b = a.view([(A,int), ('b', int)])
or:
use numpy.lib.recfunctions.rename_fields
Or change the names tuple:
a.dtype.names = ('c', 'd')
Stéfan
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] dtype field renaming
On Apr 8, 2009, at 6:18 PM, Stéfan van der Walt wrote:
2009/4/9 Pierre GM pgmdevl...@gmail.com:
for a numpy.recarray, is it possible to rename the fields in the
dtype?
Take a new view:
a = np.array([(1,1)],dtype=[('a',int),('b',int)])
b = a.view([(A,int), ('b', int)])
or:
use numpy.lib.recfunctions.rename_fields
Or change the names tuple:
a.dtype.names = ('c', 'd')
Now that's wicked neat trick ! I love it ! Faster than taking a view
for sure.
Note that rename_fields should work also w/ nested fields (not that
common, true).
___
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] Specially Constructed Arrays
Hello, I want to make an array of size sqrt(n) by sqrt(n) by 3, filled with special values. The values range from 0.0 to 3.0, starting with 0.0 at one corner and ending at 3.0 in the opposite, increasing going row by row. The value is to be encoded in each color. Because this is somewhat abstract, here's a small example (n=25), generated using the attached code (it also multiplies the number by 255 to obtain a RGB color and not messy floats) to show the concept. The real version should be done by NumPy. This is where I need help; I have no idea how to even approach the problem. [[ 0, 0, 0],[ 32, 0, 0],[ 64, 0, 0],[ 96, 0, 0],[128, 0, 0], [159, 0, 0],[191, 0, 0],[223, 0, 0],[255, 0, 0],[255, 32, 0], [255, 64, 0],[255, 96, 0],[255,128, 0],[255,159, 0],[255,191, 0], [255,223, 0],[255,255, 0],[255,255, 32],[255,255, 64],[255,255, 96], [255,255,128],[255,255,159],[255,255,191],[255,255,223],[255,255,255]] Arrays like this need to be generated quite quickly, so the per-pixel method I presented will not work. How should I do it with NumPy? Thanks, Ian def clamp(num,low,high): if numlow: return low if numhigh: return high return num def clamp_vec3(vec3,low,high): return [clamp(vec3[0],low,high), clamp(vec3[1],low,high), clamp(vec3[2],low,high)] def rndint(num): return int(round(num)) def rndint_vec3(vec3): return [rndint(vec3[0]), rndint(vec3[1]), rndint(vec3[2])] def mult_vec3(sc,vec3): return [sc*vec3[0], sc*vec3[1], sc*vec3[2]] n = 25 intensity = 0.0 step = 3.0/(n-1) surf = [] for x in xrange(n): color = clamp_vec3([intensity,intensity-1.0,intensity-2.0],0.0,1.0) color = mult_vec3(255.0,color) color = rndint_vec3(color) surf.append(color) intensity += step surf = str(surf) surf2 = for char in surf: if char != : surf2 += char print surf2 ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Blas / lapack / binary installers...
Hi Matthew, On Thu, Apr 9, 2009 at 4:44 AM, Matthew Brett matthew.br...@gmail.com wrote: Hello, Summary: is it possible to distribute, optionally or not, the blas / lapack libraries that numpy is built against, with the numpy binary installers? Yes, it is possible. We at the NIPY project have run into what seems like a recurring problem; we want to build our code against both numpy and lapack, on windows, linux and OS X. No problem of course if we've done a development install - we already needed to have blas/lapack. I am not sure I understand: why do you need blas/lapack to build projects ? Does NiPY itself uses blas/lapack ? Proposal - Make the binary installers for windows with the option to put the blas/lapack libraries in some predictable location and adapt the site.cfg file accordingly. It is possible, but it would make the installers quite big. As Gael said, a more general solution would be to use .so (well, dll), put them in a known location, so that applications can reuse it. That's something which would be great for a lot of reasons, but that's difficult to do on windows + distutils. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Blas / lapack / binary installers...
Hi, We at the NIPY project have run into what seems like a recurring problem; we want to build our code against both numpy and lapack, on windows, linux and OS X. No problem of course if we've done a development install - we already needed to have blas/lapack. I am not sure I understand: why do you need blas/lapack to build projects ? Does NiPY itself uses blas/lapack ? Yes it does... Proposal - Make the binary installers for windows with the option to put the blas/lapack libraries in some predictable location and adapt the site.cfg file accordingly. It is possible, but it would make the installers quite big. As Gael said, a more general solution would be to use .so (well, dll), put them in a known location, so that applications can reuse it. That's something which would be great for a lot of reasons, but that's difficult to do on windows + distutils. Yes, I know, hence my suggestion of something more practical in the short term. I wonder whether the installer could be: by default, smallish, with just numpy, with the option of pulling down the lapack libraries from the web on installation or, possibly optionally, large, with the lapack libraries installed by default. I was thinking, that if that were the case, then installing scipy from source might go from being very nasty (needing a compiled blas / lapack), to only fairly nasty (needing fortran), making it more accessible. Thanks a lot, Matthew ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Blas / lapack / binary installers...
On Thu, Apr 09, 2009 at 11:11:13AM +0900, David Cournapeau wrote: No problem of course if we've done a development install - we already needed to have blas/lapack. I am not sure I understand: why do you need blas/lapack to build projects ? Does NiPY itself uses blas/lapack ? NiPy uses blas/lapack in the C code, and I feel this is not an abnormal situation: as soon as you starting wanting to do vector operations in C, you end up needing blas/lapack. Proposal - Make the binary installers for windows with the option to put the blas/lapack libraries in some predictable location and adapt the site.cfg file accordingly. It is possible, but it would make the installers quite big. As Gael said, a more general solution would be to use .so (well, dll), put them in a known location, so that applications can reuse it. That's something which would be great for a lot of reasons, but that's difficult to do on windows + distutils. OK, so a short term solution might be to have a 'dev' installer that would ship the lapack libraries used to build numpy (that would address the size issue), and a long term one, which would be to expose shared libraries and headers (.so and .dll) to the applications. Does that sound like a feasible plan? I'd be interested in helping you work on part 2 of the plan, but I can't spend any time on it before fall (a set of deadlines...). Gaël ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Blas / lapack / binary installers...
Matthew Brett wrote: Yes it does... Ok. Yes, I know, hence my suggestion of something more practical in the short term. I wonder whether the installer could be: by default, smallish, with just numpy, with the option of pulling down the lapack libraries from the web on installation or, possibly optionally, large, with the lapack libraries installed by default. That's possible, but I don't know how much work it would be to make it work well. In particular, once you start using the web, you have all kind of problems with proxy and the likes. But the nsis installer certainly can support this: http://nsis.sourceforge.net/NSISdl_plug-in I would prefer that to a developer version, to avoid people having to choose which version they want. cheers, David ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
