On 10/23/2013 01:02 PM, Barrett, Brian W wrote:
On 10/22/13 10:23 AM, "Jai Dayal" <dayals...@gmail.com
<mailto:dayals...@gmail.com>> wrote:
I, for the life of me, can't understand the difference between
these two init_thread modes.
MPI_THREAD_SINGLE states that "only one thread will execute", but
MPI_THREAD_FUNNELED states "The process may be multi-threaded, but
only the main thread will make MPI calls (all MPI calls are
funneled to the main thread)."
If I use MPI_THREAD_SINGLE, and just create a bunch of pthreads
that dumbly loop in the background, the MPI library will have no
way of detecting this, nor should this have any affects on the
machine.
This is exactly the same as MPI_THREAD_FUNNELED. What exactly does
it mean with "only one thread will execute?" The openmpi library
has absolutely zero way of knowng I've spawned other pthreads, and
since these pthreads aren't actually doing MPI communication, I
fail to see how this would interfere.
Technically, if you call MPI_INIT_THREAD with MPI_THREAD_SINGLE, you
have made a promise that you will not create any other threads in your
application. There was a time where OSes shipped threaded and
non-threaded malloc, for example, so knowing that might be important
for that last bit of performance. There are also some obscure corner
cases of the memory model of some architectures where you might get
unexpected results if you made an MPI Receive call in an thread and
accessed that buffer later from another thread, which may require
memory barriers inside the implementation, so there could be some
differences between SINGLE and FUNNELED due to those barriers.
In Open MPI, we'll handle those corner cases whether you init for
SINGLE or FUNNELED, so there's really no practical difference for Open
MPI, but you're then slightly less portable.
I'm asking because I'm using an open_mpi build ontop of
infiniband, and the maximum thread mode is MPI_THREAD_SINGLE.
That doesn't seem right; which version of Open MPI are you using?
Brian
As Brian said, you aren't likely to be running on a system like Windows
98 where non-thread-safe libraries were preferred. My colleagues at
NASA insist that any properly built MPI will support MPI_THREAD_FUNNELED
by default, even when the documentation says explicit setting in
MPI_Init_thread() is mandatory. The statement which I see in OpenMPI
doc says all MPI calls must be made by the thread which calls
MPI_Init_thread. Apparently it will work if plain MPI_Init is used
instead. This theory appears to hold up for all the MPI implementations
of interest. The additional threads referred to are "inside the MPI
rank," although I suppose additional application threads not involved
with MPI are possible.
