Re: [OMPI users] 1.7.1 Hang with MPI_THREAD_MULTIPLE set

2013-06-04 Thread Jeff Squyres (jsquyres) 
On Jun 4, 2013, at 8:20 AM, W Spector  wrote:

>> Yes, this is true -- MPI_THREAD_MULITPLE support is fairly incomplete in 
>> Open MPI.
> 
> One would hope a simple MPI_Barrier call would work though...

Underneath, I am pretty sure that barrier is doing an MPI_WAITALL.  

Nathan actually just agreed to look at this bug today.

> My home linux system is nothing sophisticated.  Just a quad core I-5 on a 
> Intel DP55WB motherboard and Ubuntu Linux.  No fancy interconnects.


I'm guessing it's a double lock of some kind (i.e., a coding error in OMPI); 
not related to your network.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] 1.7.1 Hang with MPI_THREAD_MULTIPLE set

2013-06-04 Thread W Spector 

On 06/04/2013 03:23 AM, Jeff Squyres (jsquyres) wrote:

On Jun 3, 2013, at 5:06 AM, Paul Kapinos  wrote:


It is more or less well-known that MPI_THREAD_MULTIPLE disable the OpenFabric / 
InfiniBand networking in Open MPI:

http://www.open-mpi.org/faq/?category=supported-systems#thread-support
http://www.open-mpi.org/community/lists/users/2010/03/12345.php


Yes, this is true -- MPI_THREAD_MULITPLE support is fairly incomplete in Open 
MPI.


One would hope a simple MPI_Barrier call would work though...

My home linux system is nothing sophisticated.  Just a quad core I-5 on 
a Intel DP55WB motherboard and Ubuntu Linux.  No fancy interconnects.


Walter

Re: [OMPI users] 1.7.1 Hang with MPI_THREAD_MULTIPLE set

2013-06-04 Thread Jeff Squyres (jsquyres) 
On Jun 3, 2013, at 5:06 AM, Paul Kapinos  wrote:

> It is more or less well-known that MPI_THREAD_MULTIPLE disable the OpenFabric 
> / InfiniBand networking in Open MPI:
> 
> http://www.open-mpi.org/faq/?category=supported-systems#thread-support
> http://www.open-mpi.org/community/lists/users/2010/03/12345.php

Yes, this is true -- MPI_THREAD_MULITPLE support is fairly incomplete in Open 
MPI.

> On our system not only the 'openib' BTL is off, but also the IPoIB denies to 
> work, leading to error.

That's odd -- are you saying that the TCP BTL doesn't work with IPoIB?  I can't 
think of why that would be the case.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] 1.7.1 Hang with MPI_THREAD_MULTIPLE set

2013-06-03 Thread Paul Kapinos 

Hello,

It is more or less well-known that MPI_THREAD_MULTIPLE disable the OpenFabric / 
InfiniBand networking in Open MPI:


http://www.open-mpi.org/faq/?category=supported-systems#thread-support
http://www.open-mpi.org/community/lists/users/2010/03/12345.php

On our system not only the 'openib' BTL is off, but also the IPoIB denies to 
work, leading to error.


But I was able to run your programm error-free when completely disabling using 
the InfiniBand: either both processes on same node (using shared memory), or use 
"-mca btl ^openib -mca btl_tcp_if_exclude ib0,lo" parameter to the 'mpiexec' in 
order to disable InfiniBand and IPoIB.
Well; this is disappinting due to some 20x loss of performance using Gigagbit 
Ethernet, comparing the actual InfiniBand...


Note: Intel MPI support MPI_THREAD_MULTIPLE when linked with -mt_mpi (Intel and 
GCC compilers) or -lmpi_mt instead of -lmpi (other compilers). However, Intel 
MPI is not free.


Best,

Paul Kapinos



Also, I recommend to _always_ check what kinda of threading lievel you ordered 
and what did you get:

  print *, 'hello, world!', MPI_THREAD_MULTIPLE, provided







On 05/31/13 06:12, W Spector wrote:

Dear OpenMPI group,

The following trivial program hangs on the mpi_barrier call with 1.7.1.  I am
using gfortran/gcc 4.6.3 on Ubuntu linux.  OpenMPI was built with
--enable-mpi-thread-multiple support and no other options (other than --prefix).

Are there additional options we should be telling configure about?  Or have we
done something very silly?  Mpich2 works just fine...

Walter Spector


program hang
   use mpi
   implicit none

   integer :: me, npes
   integer :: mpierr, provided
   logical :: iampe0

   call mpi_init_thread (  &
   MPI_THREAD_MULTIPLE,  &
   provided,  &
   mpierr)
   print *, 'hello, world!'

! Hangs here with MPI_THREAD_MULTIPLE set...
   call mpi_barrier (MPI_COMM_WORLD, mpierr)

   call mpi_comm_rank (MPI_COMM_WORLD, me, mpierr)
   iampe0 = me == 0
   call mpi_comm_size (MPI_COMM_WORLD, npes, mpierr)
   print *, 'pe:', me, ', total comm size:', npes
   print *, 'I am ', trim (merge ('PE 0', 'not PE 0', iampe0))

   call mpi_finalize (mpierr)

end program
___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users





--
Dipl.-Inform. Paul Kapinos   -   High Performance Computing,
RWTH Aachen University, Center for Computing and Communication
Seffenter Weg 23,  D 52074  Aachen (Germany)
Tel: +49 241/80-24915



smime.p7s
Description: S/MIME Cryptographic Signature