On Mar 22, 2007, at 7:55 AM, Jeff Squyres wrote:
On Mar 15, 2007, at 12:18 PM, Michael wrote:
....
Situation: I'm compiling my code locally on a machine with just
ethernet interfaces and OpenMPI 1.1.2 that I built.
When I attempt to run that executable on a HPC machine with OpenMPI
1.1.2 and InfiniBand interfaces I get messages about "can't find
libmosal.so.0.0" -- I'm certain this wasn't happening earlier.
I can compile on this machine and run on it, even though there is no
libmosal.* in my path.
mpif90 --showme on this system gives me:
/opt/compiler/intel/compiler91/x86_64/bin/ifort -I/opt/mpi/x86_64/
intel/9.1/openmpi-1.1.4/include -pthread -I/opt/mpi/x86_64/intel/9.1/
openmpi-1.1.4/lib -L/opt/mpi/x86_64/intel/9.1/openmpi-1.1.4/lib -L/
opt/gm/lib64 -lmpi_f90 -lmpi -lorte -lopal -lgm -lvapi -lmosal -lrt -
lnuma -ldl -Wl,--export-dynamic -lnsl -lutil -ldl
Based on this output, I assume you have configured OMPI with either --
enable-static or otherwise including all plugins in libmpi.so, right?
No, I did not configure OpenMPI on this machine. I believe OpenMPI
was configured not static by the installers based on the messages and
the dependency on the missing libraries.
The issue was that some of the 1000+ nodes on this major HPC machine
were missing libraries needed for OpenMPI but because of the low
usage of OpenMPI I'm the first to discover the problem. For whatever
reason these libraries are not on the front-end machines that feed
the main system. It's always nice running OpenMPI on your own
machine but not everyone can always do that.
The way I read my experience is that OpenMPI's libmpi.so depends on
different libraries on different machines, this means that if you
don't compile static you can compile on a machine that does not have
libraries for expensive interfaces and run on another machine with
those expensive interfaces -- that's what I'm am doing now successfully.
Michael
