Robert Kern wrote:
On Sat, Mar 15, 2008 at 2:48 PM, Gnata Xavier <[EMAIL PROTECTED]> wrote:
Hi,
Numpy is great : I can see several IDL/matlab projects switching to numpy :)
However, it would be soooo nice to be able to put some OpenMP into the
numpy code.
It would be nice to be able to be able to use several CPU using the
numpy syntax ie A=sqrt(B).
Ok, we can use some inline C/C++ code but it is not so easy.
Ok, we can split the data over several python executables (one per CPU)
but A=sqrt(B) is so simple...
numpy + recent gcc with OpenMP --> :) ?
Any comments ?
Eric Jones tried to use multithreading to split the computation of
ufuncs across CPUs. Ultimately, the overhead of locking and unlocking
made it prohibitive for medium-sized arrays and only somewhat
disappointing improvements in performance for quite large arrays. I'm
not familiar enough with OpenMP to determine if this result would be
applicable to it. If you would like to try, we can certainly give you
pointers as to where to start.
Well of course if the arrays are too small it will be slower but it *is*
much faster on large arrays.
In many cases, there is no need lock/unlock : Look at A=sqrt(A) : it is
obvious to speed-up such a compuation in pure C using OpenMP. From a
simple minded point of view, I would say that somewhere in numpy, there
should be such a C loop.
Why do we really (IMHO) need that ? Because "all" the machines (even
laptops) are now multicore/cpu.
Using IDL, it is possible to develop a quite large image processing
project working on large images on a 8core machine without *any*
konwledge of semaphore/lock.
All this piece of software uses the simple syntaxe of IDL (ugly ones
compare to numpy :))
I used to be very sceptical about the performances but I had a look and
*it just works well*. It scales just nicely up to 6 cores.
Small arrays computation shall *not* be threaded, large ones should be
if we look at the multicores trend.
Any comments ?
Xavier
_______________________________________________
Numpy-discussion mailing list
Numpy-discussion@scipy.org
http://projects.scipy.org/mailman/listinfo/numpy-discussion
