Hi Ilya 1) The only thing that stands out as very different from what I do here is your configuration flag "--enable-mpi-threads".
Maybe some OpenMPI pro/developer could shed some light about this, whether that flag could be a potential source for the errors you see. Considering that when you suppress it all works fine, I would think that the "--enable-mpi-threads" is the culprit. 2) Another thing is that we never remove the loopback from the btl, i.e, we do NOT do btl_tcp_if_exclude=lo,eth0, just btl_tcp_if_exclude=eth0. I presume your choice will leave only the shared memory "sm" mechanism for intra-node communication. Not sure if this is the best choice. The OpenMPI developers may have something to say about it. 3) Other differences are that we use ssh, not rsh, but I don't think that rsh would cause a problem to you. Also, we use Infiniband (IB) instead of Ethernet for production runs. However, I tested OpenMPI with Ethernet and it works like a charm too (selecting eth1 just as you did), although not as fast as with IB. I hope this helps, Gus Correa --------------------------------------------------------------------- Gustavo Correa Lamont-Doherty Earth Observatory - Columbia University Palisades, NY, 10964-8000 - USA --------------------------------------------------------------------- ilya zelenchuk wrote:
Happy New Year, Gus! Yes, I'm using affinity. This is my openmpi-mca-params.conf file: --- # Use RSH instead SSH pls_rsh_agent=rsh # Turning on processor affinity mpi_paffinity_alone=1 # Include using eth1. btl_tcp_if_include=eth1 # Exclude using lo and eth0. btl_tcp_if_exclude=lo,eth0 --- I rebuild openmpi-1.3.3 from tar and find something interesting. If i build with this options: CC=icc CXX=icpc F77=ifort FC=ifort ./configure --prefix=/opt/testing/default/openmpi-1.3.3/ --disable-ipv6 All work's fine!!! But if I use: CC=icc CFLAGS=" -O3" CXX=icpc CXXFLAGS=" -O3" F77=ifort FFLAGS=" -O3" FC=ifort FCFLAGS=" -O3" ./configure --prefix=/opt/testing/tuning/openmpi-1.3.3/ --enable-debug --enable-mpi-threads --disable-ipv6 Then I get errors. What you think? Now I'm starting to investigate, what options cause OpenMPI broken. _______________________________________________ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
