Hi. That is really amazing.
I checked out that numexpr branch and saw some strange results when
evaluating expressions on a multi-core i7 processor.
Running the numexpr.test() yields a few 'F', which I suppose are failing
tests. I tried to let the tests finish but it takes more than 20 min, is
This is a slight digression: is there a way to have a out parameter
like semantics with numexpr. I have always used it as
a[:] = numexpr(expression)
In order to make sure the 1.6 nditer supports multithreading, I adapted
numexpr to use it. The branch which does this is here:
There is a total lack of vectorization in your code, so you are right
about the lack of vectorization.
What happens is that you see the result of the Matlab JIT compiler
speeding up the loop.
With a vectorized array expression, there will hardly be any difference.
Sturla
Den 19.07.2011
Those are very interesting examples. I think that pre-allocation is very
important, and something similar happens in Matlab if no pre-allocation is
done: it takes 3-4x longer than with pre-allocation.
The main difference is that Matlab is able to take into account a
pre-allocated array/matrix,
Den 19.07.2011 17:49, skrev Carlos Becker:
- Matlab: 0.0089
- Numpy: 0.051
- Numpy with blitz: 0.0043
Now blitz is considerably faster! Anyways, I am concerned about numpy
being much slower, in this case taking 2x the time of the previous
operation.
I guess this is because of the way
Den 20.07.2011 08:49, skrev Carlos Becker:
The main difference is that Matlab is able to take into account a
pre-allocated array/matrix, probably avoiding the creation of a
temporary and writing the results directly in the pre-allocated array.
I think this is essential to speed up numpy.
On Wednesday, July 20, 2011, Carlos Becker carlosbec...@gmail.com wrote:
Those are very interesting examples. I think that pre-allocation is very
important, and something similar happens in Matlab if no pre-allocation is
done: it takes 3-4x longer than with pre-allocation.The main difference
Hi all. Thanks for the feedback.
My point is not to start a war on matlab/numpy. This comes out of my wish to
switch from Matlab to something more appealing.
I like numpy and python, being a proper language (not like matlab scripts,
whose syntax is patched and destroyed as new versions come up).
I
Den 19.07.2011 11:05, skrev Carlos Becker:
N = 100;
tic;
for I=1:N
k = m - 0.5;
end
toc / N
m = rand(2000,2000);
Here, Matlab's JIT compiler can probably hoist the invariant out of the
loop, and just do
I=N
k = m - 0.5
Try
Den 20.07.2011 09:35, skrev Carlos Becker:
In my case, sometimes it is required to process 1k images or more, and
2x speed improvement in this case means 2 hours of processing vs 4.
Can you demonstrate that Matlab is faster than NumPy for this task?
Sturla
Wed, 20 Jul 2011 08:49:21 +0200, Carlos Becker wrote:
Those are very interesting examples. I think that pre-allocation is very
important, and something similar happens in Matlab if no pre-allocation
is done: it takes 3-4x longer than with pre-allocation. The main
difference is that Matlab is
Hi,
On Wed, Jul 20, 2011 at 2:42 AM, Chad Netzer chad.net...@gmail.com wrote:
On Tue, Jul 19, 2011 at 6:10 PM, Pauli Virtanen p...@iki.fi wrote:
k = m - 0.5
does here the same thing as
k = np.empty_like(m)
np.subtract(m, 0.5, out=k)
The memory allocation
On Tue, Jul 19, 2011 at 11:49 PM, Carlos Becker carlosbec...@gmail.com wrote:
Those are very interesting examples.
Cool.
I think that pre-allocation is very
important, and something similar happens in Matlab if no pre-allocation is
done: it takes 3-4x longer than with pre-allocation.
Can
I will be away from my computer for a week, but what I could try today
shows that Matlab JIT is doing some tricks so the results I have shown
previously for Matlab are likely to be wrong.
In this sense, it seems to be that timings are similar between numpy
and matlab if Jit tricks are
Wed, 20 Jul 2011 09:04:09 +, Pauli Virtanen wrote:
Wed, 20 Jul 2011 08:49:21 +0200, Carlos Becker wrote:
Those are very interesting examples. I think that pre-allocation is
very important, and something similar happens in Matlab if no
pre-allocation is done: it takes 3-4x longer than with
On Wed, Jul 20, 2011 at 3:57 AM, eat e.antero.ta...@gmail.com wrote:
Perhaps slightly OT, but here is something very odd going on. I would expect
the performance to be in totally different ballpark.
t=timeit.Timer('m =- 0.5', setup='import numpy as np;m =
np.ones([8092,8092],float)')
Wed, 20 Jul 2011 11:31:41 +, Pauli Virtanen wrote:
[clip]
There is a sharp order-of-magnitude change of speed in malloc+memset of
an array, which is not present in memset itself. (This is then also
reflected in the Numpy performance -- floating point operations probably
don't cost much
I think this is essential to speed up numpy. Maybe numexpr could handle this
in the future? Right now the general use of numexpr is result =
numexpr.evaluate(whatever), so the same problem seems to be there.
With this I am not saying that numpy is not worth it, just that for many
On Wed, Jul 20, 2011 at 5:52 PM, srean srean.l...@gmail.com wrote:
I think this is essential to speed up numpy. Maybe numexpr could handle
this in the future? Right now the general use of numexpr is result =
numexpr.evaluate(whatever), so the same problem seems to be there.
With this I am
Hi, I started with numpy a few days ago. I was timing some array operations
and found that numpy takes 3 or 4 times longer than Matlab on a simple
array-minus-scalar operation.
This looks as if there is a lack of vectorization, even though this is just
a guess. I hope this is not reposting. I
Tue, 19 Jul 2011 11:05:18 +0200, Carlos Becker wrote:
Hi, I started with numpy a few days ago. I was timing some array
operations and found that numpy takes 3 or 4 times longer than Matlab on
a simple array-minus-scalar operation.
This looks as if there is a lack of vectorization, even though
Hi Pauli, thanks for the quick answer.
Is there a way to check the optimization flags of numpy after
installation?
I am away of a matlab installation now, but I remember I saw a single
processor active with matlab. I will check it again soon
Thanks!
El 19/07/2011, a las 13:10, Pauli
I made more tests with the same operation, restricting Matlab to use a
single processing unit. I got:
- Matlab: 0.0063 sec avg
- Numpy: 0.026 sec avg
- Numpy with weave.blitz: 0.0041
Note that weave.blitz is even faster than Matlab (slightly).
I tried on an older computer, and I got similar
On Tue, Jul 19, 2011 at 9:49 AM, Carlos Becker carlosbec...@gmail.comwrote:
I made more tests with the same operation, restricting Matlab to use a
single processing unit. I got:
- Matlab: 0.0063 sec avg
- Numpy: 0.026 sec avg
- Numpy with weave.blitz: 0.0041
Note that weave.blitz is even
On Tue, Jul 19, 2011 at 11:19 AM, Charles R Harris
charlesr.har...@gmail.com wrote:
On Tue, Jul 19, 2011 at 9:49 AM, Carlos Becker carlosbec...@gmail.comwrote:
I made more tests with the same operation, restricting Matlab to use a
single processing unit. I got:
- Matlab: 0.0063 sec avg
Hi, everything was run on linux.
I am using numpy 2.0.0.dev-64fce7c, but I tried an older version (cannot
remember which one now) and saw similar results.
Matlab is R2011a, and I used taskset to assign its process to a single core.
Linux is 32-bit, on Intel Core i7-2630QM.
Besides the
On Tue, Jul 19, 2011 at 4:05 AM, Carlos Becker carlosbec...@gmail.com wrote:
Hi, I started with numpy a few days ago. I was timing some array operations
and found that numpy takes 3 or 4 times longer than Matlab on a simple
array-minus-scalar operation.
Doing these kinds of timings correctly
On Tue, Jul 19, 2011 at 2:27 PM, Carlos Becker carlosbec...@gmail.com wrote:
Hi, everything was run on linux.
Placing parentheses around the scalar multipliers shows that it seems to
have to do with how expressions are handled, is there sometihng that can be
done about this so that numpy can
On Tue, 19 Jul 2011 17:49:14 +0200, Carlos Becker wrote:
I made more tests with the same operation, restricting Matlab to use a
single processing unit. I got:
- Matlab: 0.0063 sec avg
- Numpy: 0.026 sec avg
- Numpy with weave.blitz: 0.0041
To check if it's an issue with building without
For such expressions you should try numexpr package: It allows the same type of
optimisation as Matlab does: run a single loop over the matrix elements
instead of repetitive loops and intermediate objects creation.
Nadav
Besides the matlab/numpy comparison, I think that there is an
Carlos Becker wrote:
Besides the matlab/numpy comparison, I think that there is an inherent
problem with how expressions are handled, in terms of efficiency.
For instance, k = (m - 0.5)*0.3 takes 52msec average here (2000x2000
array), while k = (m - 0.5)*0.3*0.2 takes 0.079, and k = (m -
Thanks Chad for the explanation on those details. I am new to python and I
still have a lot to learn, this was very useful.
Now I get similar results between matlab and numpy when I re-use the memory
allocated for m with 'm -= 0.5'.
However, if I don't, I obtain this 4x penalty with numpy, even
On Tue, Jul 19, 2011 at 3:35 PM, Carlos Becker carlosbec...@gmail.com wrote:
Thanks Chad for the explanation on those details. I am new to python and I
However, if I don't, I obtain this 4x penalty with numpy, even with the
8092x8092 array. Would it be possible to do k = m - 0.5 and
On Jul 19, 2011, at 3:15 PM, Chad Netzer wrote:
%python
import timeit
import numpy as np
t=timeit.Timer('k = m - 0.5', setup='import numpy as np;m =
np.ones([8092,8092],float); k = np.zeros(m.size, m.dtype)')
np.mean(t.repeat(repeat=10, number=1))
0.58557529449462886
t=timeit.Timer('k
On Tue, 19 Jul 2011 17:15:47 -0500, Chad Netzer wrote:
On Tue, Jul 19, 2011 at 3:35 PM, Carlos Becker carlosbec...@gmail.com
[clip]
However, if I don't, I obtain this 4x penalty with numpy, even with the
8092x8092 array. Would it be possible to do k = m - 0.5 and
pre-alllocate k such that
On Tue, Jul 19, 2011 at 6:10 PM, Pauli Virtanen p...@iki.fi wrote:
k = m - 0.5
does here the same thing as
k = np.empty_like(m)
np.subtract(m, 0.5, out=k)
The memory allocation (empty_like and the subsequent deallocation)
costs essentially nothing, and there are no
36 matches
Mail list logo
