Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Robert Kern
On Tue, Jan 20, 2009 at 20:57, Neal Becker ndbeck...@gmail.com wrote: I see the problem. Thanks for the great profiler! You ought to make this more widely known. http://pypi.python.org/pypi/line_profiler -- Robert Kern I have come to believe that the whole world is an enigma, a harmless

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-21 Thread Francesc Alted
A Tuesday 20 January 2009, Andrew Collette escrigué: Works much, much better with the current svn version. :) Numexpr now outperforms everything except the simple technique, and then only for small data sets. Correct. This is because of the cost of parsing the expression and initializing the

[Numpy-discussion] numpy and bitwise arrays?

2009-01-21 Thread Muhammad Alkarouri
Hi everyone, I need to use a bitwise array, and I wanted to check what is the common practice using numpy. I expect to read binary strings (like '001101') of equal length from file and to save and manipulate them using numpy. I know that numpy have good implementations of bitwise

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
T J wrote: On Tue, Jan 20, 2009 at 6:57 PM, Neal Becker ndbeck...@gmail.com wrote: It seems the big chunks of time are used in data conversion between numpy and my own vectors classes. Mine are wrappers around boost::ublas. The conversion must be falling back on a very inefficient method

[Numpy-discussion] savetxt and blanks

2009-01-21 Thread Nils Wagner
Hi all, Is it possible to add a certain number of blanks behind the second column in connection with savetxt ? from numpy.random import rand from numpy import savetxt A = rand(100,2) savetxt('noblanks.dat',A,fmt='%10.2f %10.2f') Nils ___

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
Robert Kern wrote: On Tue, Jan 20, 2009 at 20:57, Neal Becker ndbeck...@gmail.com wrote: I see the problem. Thanks for the great profiler! You ought to make this more widely known. I'll be making a release shortly. It seems the big chunks of time are used in data conversion between

Re: [Numpy-discussion] numpy and bitwise arrays?

2009-01-21 Thread Stéfan van der Walt
Hi Muhammad 2009/1/21 Muhammad Alkarouri malkaro...@yahoo.co.uk: I need to use a bitwise array, and I wanted to check what is the common practice using numpy. You can also take a look at Ilan Schnell's bitarray: http://pypi.python.org/pypi/bitarray/ Cheers Stéfan

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Sturla Molden
On 1/21/2009 1:27 PM, Neal Becker wrote: It might if I had used this for all of my c++ code, but I have a big library of c++ wrapped code that doesn't use pyublas. Pyublas takes numpy objects from python and allows the use of c++ ublas on it (without conversion). If you can get a pointer

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Sturla Molden
On 1/21/2009 2:38 PM, Sturla Molden wrote: If you can get a pointer (as integer) to your C++ data, and the shape and dtype is known, you may use this (rather unsafe) 'fromaddress' hack: And opposite, if you need to get the address referenced to by an ndarray, you can do this: def

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Ravi
Hi Neal, On Wednesday 21 January 2009 07:27:04 Neal Becker wrote: It might if I had used this for all of my c++ code, but I have a big library of c++ wrapped code that doesn't use pyublas.  Pyublas takes numpy objects from python and allows the use of c++ ublas on it (without conversion).

Re: [Numpy-discussion] numpy and bitwise arrays?

2009-01-21 Thread Muhammad Alkarouri
--- On Wed, 21/1/09, Stéfan van der Walt ste...@sun.ac.za wrote: From: Stéfan van der Walt ste...@sun.ac.za ... You can also take a look at Ilan Schnell's bitarray: http://pypi.python.org/pypi/bitarray/ Looks good to me. Thanks for the suggestion. Muhammad

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
Ravi wrote: Hi Neal, On Wednesday 21 January 2009 07:27:04 Neal Becker wrote: It might if I had used this for all of my c++ code, but I have a big library of c++ wrapped code that doesn't use pyublas.  Pyublas takes numpy objects from python and allows the use of c++ ublas on it (without

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
Ravi wrote: Hi Neal, On Wednesday 21 January 2009 07:27:04 Neal Becker wrote: It might if I had used this for all of my c++ code, but I have a big library of c++ wrapped code that doesn't use pyublas.  Pyublas takes numpy objects from python and allows the use of c++ ublas on it (without

Re: [Numpy-discussion] numpy and bitwise arrays?

2009-01-21 Thread Travis E. Oliphant
Muhammad Alkarouri wrote: --- On Wed, 21/1/09, Stéfan van der Walt ste...@sun.ac.za wrote: From: Stéfan van der Walt ste...@sun.ac.za ... You can also take a look at Ilan Schnell's bitarray: http://pypi.python.org/pypi/bitarray/ Looks good to me. Thanks for the

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Ravi
On Wednesday 21 January 2009 10:22:36 Neal Becker wrote: [http://mail.python.org/pipermail/cplusplus-sig/2008-October/013825.html Thanks for reminding me about this! Do you have a current version of the code?  I grabbed the files from the above message, but I see some additional subsequent

Re: [Numpy-discussion] strange multiplication behavior with numpy.float64 and ndarray subclass

2009-01-21 Thread Pierre GM
On Jan 21, 2009, at 11:34 AM, Darren Dale wrote: I have a simple test script here that multiplies an ndarray subclass with another number. Can anyone help me understand why each of these combinations returns a new instance of MyArray: mine = MyArray() print type(np.float32(1)*mine)

Re: [Numpy-discussion] strange multiplication behavior with numpy.float64 and ndarray subclass

2009-01-21 Thread Darren Dale
On Wed, Jan 21, 2009 at 11:43 AM, Pierre GM pgmdevl...@gmail.com wrote: On Jan 21, 2009, at 11:34 AM, Darren Dale wrote: I have a simple test script here that multiplies an ndarray subclass with another number. Can anyone help me understand why each of these combinations returns a new

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Christopher Barker
Neal Becker wrote: I tried a little experiment, implementing some code in numpy It sounds like you've found your core issue, but a couple comments: from numpy import * I'm convinced that import * is a bad idea. I think the standard syntax is now import numpy as np from math import pi

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Wes McKinney
Robert-- this is a great little piece of code, I already think it will be a part of my workflow. However, I seem to be getting negative % time taken on the more time consuming lines, perhaps getting some overflow? Thanks a lot, Wes On Wed, Jan 21, 2009 at 3:23 AM, Robert Kern

[Numpy-discussion] failure

2009-01-21 Thread Gideon Simpson
Installing on a Sun machine with Red Hat linux, I got the following error: == FAIL: test_umath.TestComplexFunctions.test_against_cmath -- Traceback (most

Re: [Numpy-discussion] ANN: Numexpr 1.1, an efficient array evaluator

2009-01-21 Thread Andrew Collette
Hi, I get identical results for both shapes now; I manually removed the numexpr-1.1.1.dev-py2.5-linux-i686.egg folder in site-packages and reinstalled. I suppose there must have been a stale set of files somewhere. Andrew Collette On Wed, Jan 21, 2009 at 3:41 AM, Francesc Alted

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
Ravi wrote: On Wednesday 21 January 2009 10:22:36 Neal Becker wrote: [http://mail.python.org/pipermail/cplusplus-sig/2008- October/013825.html Thanks for reminding me about this! Do you have a current version of the code?  I grabbed the files from the above message, but I see some

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
Ravi wrote: On Wednesday 21 January 2009 10:22:36 Neal Becker wrote: [http://mail.python.org/pipermail/cplusplus-sig/2008- October/013825.html Thanks for reminding me about this! Do you have a current version of the code?  I grabbed the files from the above message, but I see some

[Numpy-discussion] genfromtxt

2009-01-21 Thread Brent Pedersen
hi, i'm using the new genfromtxt stuff in numpy svn, looks great pierre any who contributed. is there a way to have the header commented and still be able to have it recognized as the header? e.g. #gender age weight M 21 72.10 F 35 58.33 M 33 21.99 if i use np.loadtxt or

Re: [Numpy-discussion] genfromtxt

2009-01-21 Thread Pierre GM
Brent, Currently, no, you won't be able to retrieve the header if it's commented. I'll see what I can do. P. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Ravi
On Wednesday 21 January 2009 13:55:49 Neal Becker wrote: I'm only interested in simple strided 1-d vectors.  In that case, I think your code already works.  If you have c++ code using the iterator interface, the iterators dereference will  use (*array )[index].  This will use operator[], which

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Neal Becker
Ravi wrote: On Wednesday 21 January 2009 13:55:49 Neal Becker wrote: I'm only interested in simple strided 1-d vectors.  In that case, I think your code already works.  If you have c++ code using the iterator interface, the iterators dereference will  use (*array )[index].  This will use

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Ravi
On Wednesday 21 January 2009 14:57:59 Neal Becker wrote: ublas::vectorT func (numpy::array_from_pyT::type const) But not for a function that modifies it arg in-place ( instead of const): void func (numpy::array_from_pyT::type ) Use void func

Re: [Numpy-discussion] line_profiler suggestion

2009-01-21 Thread Robert Kern
On Wed, Jan 21, 2009 at 15:13, Neal Becker ndbeck...@gmail.com wrote: Would be handy to not have to add/remove @profile to switch between profiling/normal operation. When run without profiler, @profile is redefined to do nothing. Possible? I could add a --noop flag to kernprof, which

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Robert Kern
On Wed, Jan 21, 2009 at 12:13, Wes McKinney wesmck...@gmail.com wrote: Robert-- this is a great little piece of code, I already think it will be a part of my workflow. However, I seem to be getting negative % time taken on the more time consuming lines, perhaps getting some overflow? That's

Re: [Numpy-discussion] line_profiler suggestion

2009-01-21 Thread Robert Kern
On Wed, Jan 21, 2009 at 15:35, Neal Becker ndbeck...@gmail.com wrote: Robert Kern wrote: On Wed, Jan 21, 2009 at 15:13, Neal Becker ndbeck...@gmail.com wrote: Would be handy to not have to add/remove @profile to switch between profiling/normal operation. When run without profiler, @profile

[Numpy-discussion] No tests found for numpy 1.2.0

2009-01-21 Thread Brian Granger
Just installed numpy 1.2.0 and got this: $ python -c 'import numpy; numpy.test()' Running unit tests for numpy-1.2.0-py2.5-macosx-10.5-i386.egg.numpy NumPy version 1.2.0 NumPy is installed in /Users/bgranger/Library/Python/2.5/site-packages/numpy-1.2.0-py2.5-macosx-10.5-i386.egg/numpy Python

Re: [Numpy-discussion] No tests found for numpy 1.2.0

2009-01-21 Thread Robert Kern
On Wed, Jan 21, 2009 at 22:24, Brian Granger ellisonbg@gmail.com wrote: Just installed numpy 1.2.0 and got this: $ python -c 'import numpy; numpy.test()' Running unit tests for numpy-1.2.0-py2.5-macosx-10.5-i386.egg.numpy easy_install sets the executable bit on files and nose ignores

Re: [Numpy-discussion] genfromtxt

2009-01-21 Thread Pierre GM
Brent, Mind trying r6330 and let me know if it works for you ? Make sure that you use names=True to detect a header. P. ___ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion

Re: [Numpy-discussion] genfromtxt

2009-01-21 Thread Brent Pedersen
On Wed, Jan 21, 2009 at 9:39 PM, Pierre GM pgmdevl...@gmail.com wrote: Brent, Mind trying r6330 and let me know if it works for you ? Make sure that you use names=True to detect a header. P. yes, works perfectly. thanks! -brent ___ Numpy-discussion

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Hanni Ali
I have been using your profiler extensively and it has contributed to my achieving significant improvements in the application I work on largely due to the usefulness of the line by line breakdown enabling me to easily select the next part of code to work on optimizing. So firstly many thanks for

Re: [Numpy-discussion] python numpy code many times slower than c++

2009-01-21 Thread Robert Kern
On Thu, Jan 22, 2009 at 01:46, Hanni Ali hanni@gmail.com wrote: I have been using your profiler extensively and it has contributed to my achieving significant improvements in the application I work on largely due to the usefulness of the line by line breakdown enabling me to easily select