Hello,
I don't want to get too much off topic in this reply but you're
brigning out a point here. I am unable to run mpi apps on the AMD64 platform
with the regular exporting of $LD_LIBRARY_PATH and $PATH, this is why I have no
choice but to revert to using the --prefix approach. Here are a few execution
examples to demonstrate my point:
kyron@headless ~ $ /usr/lib64/openmpi/1.0.2-gcc-4.1/bin/mpirun --prefix
/usr/lib64/openmpi/1.0.2-gcc-4.1/ -np 2 ./a.out
./a.out: error while loading shared libraries: libmpi.so.0: cannot open shared
object file: No such file or directory
kyron@headless ~ $ /usr/lib64/openmpi/1.0.2-gcc-4.1/bin/mpirun --prefix
/usr/lib64/openmpi/1.0.2-gcc-4.1/lib64/ -np 2 ./a.out
[headless:10827] pls:rsh: execv failed with errno=2
[headless:10827] ERROR: A daemon on node localhost failed to start as expected.
[headless:10827] ERROR: There may be more information available from
[headless:10827] ERROR: the remote shell (see above).
[headless:10827] ERROR: The daemon exited unexpectedly with status 255.
kyron@headless ~ $ cat opmpi64.sh
#!/bin/bash
MPI_BASE='/usr/lib64/openmpi/1.0.2-gcc-4.1'
export PATH=$PATH:${MPI_BASE}/bin
LD_LIBRARY_PATH=${MPI_BASE}/lib64
kyron@headless ~ $ . opmpi64.sh
kyron@headless ~ $ mpirun -np 2 ./a.out
./a.out: error while loading shared libraries: libmpi.so.0: cannot open shared
object file: No such file or directory
kyron@headless ~ $
Eric
Le vendredi 16 juin 2006 10:31, Pak Lui a écrit :
> Hi, I noticed your prefix set to the lib dir, can you try without the
> lib64 part and rerun?
>
> Eric Thibodeau wrote:
> > Hello everyone,
> >
> > Well, first off, I hope this problem I am reporting is of some validity,
> > I tried finding simmilar situations off Google and the mailing list but
> > came up with only one reference [1] which seems invalid in my case since
> > all executions are local (naïve assumptions that it makes a difference
> > on the calling stack). I am trying to run asimple HelloWorld using
> > OpenMPI 1.0.2 on an AMD64 machine and a Sun Enterprise (12 procs)
> > machine. In both cases I get the following error:
> >
> > pls:rsh: execv failed with errno=2
> >
> > Here is the mpirun -d trace when running my HelloWorld (on AMD64):
> >
> > kyron@headless ~ $ mpirun -d --prefix
> > /usr/lib64/openmpi/1.0.2-gcc-4.1/lib64/ -np 4 ./hello
> >
> > [headless:10461] procdir: (null)
> >
> > [headless:10461] jobdir: (null)
> >
> > [headless:10461] unidir:
> > /tmp/openmpi-sessions-kyron@headless_0/default-universe
> >
> > [headless:10461] top: openmpi-sessions-kyron@headless_0
> >
> > [headless:10461] tmp: /tmp
> >
> > [headless:10461] [0,0,0] setting up session dir with
> >
> > [headless:10461] tmpdir /tmp
> >
> > [headless:10461] universe default-universe-10461
> >
> > [headless:10461] user kyron
> >
> > [headless:10461] host headless
> >
> > [headless:10461] jobid 0
> >
> > [headless:10461] procid 0
> >
> > [headless:10461] procdir:
> > /tmp/openmpi-sessions-kyron@headless_0/default-universe-10461/0/0
> >
> > [headless:10461] jobdir:
> > /tmp/openmpi-sessions-kyron@headless_0/default-universe-10461/0
> >
> > [headless:10461] unidir:
> > /tmp/openmpi-sessions-kyron@headless_0/default-universe-10461
> >
> > [headless:10461] top: openmpi-sessions-kyron@headless_0
> >
> > [headless:10461] tmp: /tmp
> >
> > [headless:10461] [0,0,0] contact_file
> > /tmp/openmpi-sessions-kyron@headless_0/default-universe-10461/universe-setup.txt
> >
> > [headless:10461] [0,0,0] wrote setup file
> >
> > [headless:10461] spawn: in job_state_callback(jobid = 1, state = 0x1)
> >
> > [headless:10461] pls:rsh: local csh: 0, local bash: 1
> >
> > [headless:10461] pls:rsh: assuming same remote shell as local shell
> >
> > [headless:10461] pls:rsh: remote csh: 0, remote bash: 1
> >
> > [headless:10461] pls:rsh: final template argv:
> >
> > [headless:10461] pls:rsh: /usr/bin/ssh <template> orted --debug
> > --bootproxy 1 --name <template> --num_procs 2 --vpid_start 0 --nodename
> > <template> --universe kyron@headless:default-universe-10461 --nsreplica
> > "0.0.0;tcp://142.137.135.124:37657;tcp://192.168.1.1:37657" --gprreplica
> > "0.0.0;tcp://142.137.135.124:37657;tcp://192.168.1.1:37657"
> > --mpi-call-yield 0
> >
> > [headless:10461] pls:rsh: launching on node localhost
> >
> > [headless:10461] pls:rsh: oversubscribed -- setting mpi_yield_when_idle
> > to 1 (1 4)
> >
> > [headless:10461] pls:rsh: localhost is a LOCAL node
> >
> > [headless:10461] pls:rsh: reset PATH:
> > /usr/lib64/openmpi/1.0.2-gcc-4.1/lib64/bin:/usr/local/bin:/usr/bin:/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.1.1:/opt/c3-4:/usr/qt/3/bin:/usr/lib64/openmpi/1.0.2-gcc-4.1/bin
> >
> > [headless:10461] pls:rsh: reset LD_LIBRARY_PATH:
> > /usr/lib64/openmpi/1.0.2-gcc-4.1/lib64/lib
> >
> > [headless:10461] pls:rsh: changing to directory /home/kyron
> >
> > [headless:10461] pls:rsh: executing: orted --debug --bootproxy 1 --name
> > 0.0.1 --num_procs 2 --vpid_start 0 --nodename localhost --universe
> > kyron@headless:default-universe-10461 --nsreplica
> > "0.0.0;tcp://142.137.135.124:37657;tcp://192.168.1.1:37657" --gprreplica
> > "0.0.0;tcp://142.137.135.124:37657;tcp://192.168.1.1:37657"
> > --mpi-call-yield 1
> >
> > [headless:10461] pls:rsh: execv failed with errno=2
> >
> > [headless:10461] sess_dir_finalize: proc session dir not empty - leaving
> >
> > [headless:10461] spawn: in job_state_callback(jobid = 1, state = 0xa)
> >
> > [headless:10461] sess_dir_finalize: proc session dir not empty - leaving
> >
> > [headless:10461] sess_dir_finalize: proc session dir not empty - leaving
> >
> > [headless:10461] sess_dir_finalize: proc session dir not empty - leaving
> >
> > [headless:10461] spawn: in job_state_callback(jobid = 1, state = 0x9)
> >
> > [headless:10461] ERROR: A daemon on node localhost failed to start as
> > expected.
> >
> > [headless:10461] ERROR: There may be more information available from
> >
> > [headless:10461] ERROR: the remote shell (see above).
> >
> > [headless:10461] ERROR: The daemon exited unexpectedly with status 255.
> >
> > [headless:10461] sess_dir_finalize: found proc session dir empty - deleting
> >
> > [headless:10461] sess_dir_finalize: found job session dir empty - deleting
> >
> > [headless:10461] sess_dir_finalize: found univ session dir empty - deleting
> >
> > [headless:10461] sess_dir_finalize: top session dir not empty - leaving
> >
> > The two platforms are very different, one is AMD64 (dual Opteron) with
> > GCC 4.1.1 (Gentoo), the other is SUN OS 5.8 with GCC 3.4.2. OpenMPI was
> > compiled with the following options (extracted from the config.status):
> >
> > AMD64:
> >
> > Open MPI config.status 1.0.2
> >
> > configured by ./configure, generated by GNU Autoconf 2.59,
> >
> > with options \"'--prefix=/usr' '--host=x86_64-pc-linux-gnu'
> > '--mandir=/usr/share/man' '--infodir=/usr/share/info'
> > '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib'
> > '--prefix=/usr/lib64/openmpi/1.0.2-gcc-4.1'
> > '--datadir=/usr/share/openmpi/1.0.2-gcc-4.1'
> > '--program-suffix=-1.0.2-gcc-4.1'
> > '--sysconfdir=/etc/openmpi/1.0.2-gcc-4.1'
> > '--enable-pretty-print-stacktrace'
> > '--libdir=/usr/lib64/openmpi/1.0.2-gcc-4.1/lib64'
> > '--build=x86_64-pc-linux-gnu' '--cache-file' 'config.cache'
> > 'CFLAGS=-march=opteron -O2 -pipe -ftracer -fprefetch-loop-arrays
> > -mfpmath=sse -ffast-math -ftree-vectorize -floop-optimize2'
> > 'CXXFLAGS=-march=opteron -O2 -pipe -ftracer -fprefetch-loop-arrays
> > -mfpmath=sse -ffast-math -ftree-vectorize -floop-optimize2' 'LDFLAGS=
> > -Wl,-z,-noexecstack' 'build_alias=x86_64-pc-linux-gnu'
> > 'host_alias=x86_64-pc-linux-gnu' --enable-ltdl-convenience\"
> >
> > SUN 5.8:
> >
> > Open MPI config.status 1.0.2
> >
> > configured by ./configure, generated by GNU Autoconf 2.59,
> >
> > with options
> > \"'--prefix=/export/lca/home/lca0/etudiants/ac38820/openmpi'
> > '--enable-pretty-print-stacktrace' 'CFLAGS=-mv8plus' 'CXXFLAGS=-mv8plus'
> > --enable-ltdl-convenience\"
> >
> > x86 (as a working reference, configure options should be close to
> > identical as the AMD64):
> >
> > Open MPI config.status 1.0.2
> >
> > configured by ./configure, generated by GNU Autoconf 2.59,
> >
> > with options \"'--prefix=/usr' '--host=i686-pc-linux-gnu'
> > '--mandir=/usr/share/man' '--infodir=/usr/share/info'
> > '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib'
> > '--prefix=/usr/lib/openmpi/1.0.2-gcc-4.1'
> > '--datadir=/usr/share/openmpi/1.0.2-gcc-4.1'
> > '--program-suffix=-1.0.2-gcc-4.1'
> > '--sysconfdir=/etc/openmpi/1.0.2-gcc-4.1'
> > '--enable-pretty-print-stacktrace' '--build=i686-pc-linux-gnu'
> > '--cache-file' 'config.cache' 'CFLAGS=-march=nocona -O2 -pipe
> > -fomit-frame-pointer' 'CXXFLAGS=-march=nocona -O2 -pipe
> > -fomit-frame-pointer' 'LDFLAGS= -Wl,-z,-noexecstack'
> > 'build_alias=i686-pc-linux-gnu' 'host_alias=i686-pc-linux-gnu'
> > --enable-ltdl-convenience\"
> >
> > Any help would be greatly appreciated.
> >
> > Thanks.
> >
> > [1] http://gridengine.sunsource.net/servlets/ReadMsg?list=users&msgNo=15775
> >
> > --
> >
> > Eric Thibodeau
> >
> > Neural Bucket Solutions Inc.
> >
> > T. (514) 736-1436
> >
> > C. (514) 710-0517
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > users mailing list
> > us...@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
--
Eric Thibodeau
Neural Bucket Solutions Inc.
T. (514) 736-1436
C. (514) 710-0517
