On 5/14/2011 8:12 AM, David Mertens wrote:
Hello everybody -
I recently pulled in the pthreading patch that John just pushed. I am
blown away that John was able to go deep into the internals and make
automated pthreading work without any additional user effort. Amazing!
A lot of the effort was just cleaning up and adding an interface to
previous work by Christian Soeller for the add_threading_magic function.
The standard tests compiled and ran on my laptop. I then ran the tests
with forced pthreading using (in bash)
$ PDL_AUTOPTHREAD_TARG=4 PDL_AUTOPTHREAD_SIZE=0 make --no-print-directory test
All of these passed, too. However, PLplot's tests *should not* have
passed. PLplot's C-implementation uses a global state machine and the
low-level plotting commands thread over piddle arguments. I don't
believe it's possible for that to be pthread-safe. However, there are
no tests to check against this, since until now all PDL threading has
run on a single pthread. I don't believe this is likely to cause
trouble with PLplot from most use cases. John explicitly recommends
against a pthread size of 0, and a pthread size of 1 refers to 1
megabyte of data. (Who would try plotting megabytes of data on one
plot?) However, almost certainly somebody is going to set a pthread
size of 0 in their environment variables, just for fun, and then plots
using a PLOTTYPE of POINTS and with various colors will inexplicably
begin to fail.
To help see what is going on with PLplot, It would help to break it down
to a small test case where the input data has some extra threaded
dimensions, then run with the PDL_AUTOPTHREAD_TARG=4
PDL_AUTOPTHREAD_SIZE=0 arguments, and then call the
get_autopthread_actual() to verify the number of pthreads that were
actually executed.
When developing the processor multi-threading patch, I found that the
fft and the pnmout functions were not thread-safe. They would sometimes
crash (segfault) when running the test cases, but not always. It is
possible that PLplot test cases would have to be run multiple times to
see any problems.
In any case, if PLplot is known to be not thread-safe, then it is best
to put the NoPthread flag in the PPcode for the PLplot package so that
pthreading won't be attempted. This is what was done for the fft and the
pnmout functions.
Also, some clarifications from your message above:
John explicitly recommends
against a pthread size of 0, and a pthread size of 1 refers to 1
megabyte of data.
There is nothing wrong with running with PDL_AUTOPTHREAD_SIZE=0. The
will make code attempt to do multiple pthreads on any size PDL. Since
there is some overhead involved in creating multiple pthreads, there
probably won't be a speed benefit to doing multiple pthreads small PDL
arrays. However, I haven't done a whole lot of benchmarking to verify this.
Also the PDL_AUTOPTHREAD_SIZE refers to the number of elements in the
PDL, not the size of the PDL in bytes. For example, doing a
zeroes(5000,5000) will create an double-precision array with about 24Meg
Elements, but with a size in memory of 24*8 = 190Mbyes
_______________________________________________
PDL-porters mailing list
PDL-porters@jach.hawaii.edu
http://mailman.jach.hawaii.edu/mailman/listinfo/pdl-porters
