Re: [OMPI users] relocating an installation
To be clear, --prefix and OPAL_PREFIX do two different things: 1. "mpirun --prefix" and "/abs/path/to/mpirun" (both documented in mpirun(1)) are used to set PATH and LD_LIBRARY_PATH on remote nodes when invoked via ssh/rsh. It's a way of not having to set your shell startup files to point to a specific Open MPI installation on remote nodes. This is helpful for users who tend to swap between multiple Open MPI installations. 2. OPAL_PREFIX overrides the compiled-in location in Open MPI binaries and libraries for where it should find things like plugins and help files. It is intended for use when you relocate Open MPI installations. These two can be used in conjunction with each other, if desired, but they are intended for two different purposes. > On Apr 10, 2019, at 9:06 AM, Reuti wrote: > > >> Am 10.04.2019 um 14:57 schrieb Dave Love : >> >> In fact, setting OPAL_PREFIX doesn't work for a relocated tree (with >> OMPI 1.10 or 3.0). You also need $OPAL_PREFIX/lib and >> $OPAL_PREFIX/lib/openmpi on LD_LIBRARY_PATH (assuming >> $MPI_LIB=$MPI_HOME/lib): >> >> $ OPAL_PREFIX=$(pwd)/usr/lib64/openmpi3 ./usr/lib64/openmpi3/bin/mpirun >> mpirun true >> ./usr/lib64/openmpi3/bin/mpirun: error while loading shared libraries: >> libopen-rte.so.40: cannot open shared object file: No such file or directory > > Does export this OPAL_PREFIX also to all child processes? For me using export > OPAL_PREFIX=… beforehand worked up to now. > > -- Reuti > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users -- Jeff Squyres jsquy...@cisco.com ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
> Am 10.04.2019 um 14:57 schrieb Dave Love : > > In fact, setting OPAL_PREFIX doesn't work for a relocated tree (with > OMPI 1.10 or 3.0). You also need $OPAL_PREFIX/lib and > $OPAL_PREFIX/lib/openmpi on LD_LIBRARY_PATH (assuming $MPI_LIB=$MPI_HOME/lib): > > $ OPAL_PREFIX=$(pwd)/usr/lib64/openmpi3 ./usr/lib64/openmpi3/bin/mpirun > mpirun true > ./usr/lib64/openmpi3/bin/mpirun: error while loading shared libraries: > libopen-rte.so.40: cannot open shared object file: No such file or directory Does export this OPAL_PREFIX also to all child processes? For me using export OPAL_PREFIX=… beforehand worked up to now. -- Reuti ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
In fact, setting OPAL_PREFIX doesn't work for a relocated tree (with OMPI 1.10 or 3.0). You also need $OPAL_PREFIX/lib and $OPAL_PREFIX/lib/openmpi on LD_LIBRARY_PATH (assuming $MPI_LIB=$MPI_HOME/lib): $ OPAL_PREFIX=$(pwd)/usr/lib64/openmpi3 ./usr/lib64/openmpi3/bin/mpirun mpirun true ./usr/lib64/openmpi3/bin/mpirun: error while loading shared libraries: libopen-rte.so.40: cannot open shared object file: No such file or directory ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
"Jeff Squyres (jsquyres) via users" writes: > Reuti's right. > > Sorry about the potentially misleading use of "--prefix" -- we > basically inherited that CLI option from a different MPI > implementation (i.e., people asked for it). So we were locked into > that meaning for the "--prefix" CLI options. Reading more closely, I see it's only for remote hosts anyhow, but I'd be surprised if only binaries and libraries were relocated from one tree, and only hosts that somehow were distinguished from the one invoking mpirun (et al). Regardless, is there some problem just documenting the relevant variables in the obvious places? ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
Reuti writes: >> It should be documented. > > There is this FAQ entry: > > https://www.open-mpi.org/faq/?category=building#installdirs For what it's worth, I looked under "running" in the FAQ, as I was after a runtime switch. I expect FAQs to point to the actual documentation, though, and an environment variables section of the man pages seems the right place. [I know you're only providing info, thanks.] ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
Reuti's right. Sorry about the potentially misleading use of "--prefix" -- we basically inherited that CLI option from a different MPI implementation (i.e., people asked for it). So we were locked into that meaning for the "--prefix" CLI options. > On Apr 9, 2019, at 9:14 AM, Reuti wrote: > >> >> Am 09.04.2019 um 14:52 schrieb Dave Love : >> >> Reuti writes: >> >>> export OPAL_PREFIX= >>> >>> to point it to the new location of installation before you start `mpiexec`. >> >> Thanks; that's now familiar, and I don't know how I missed it with >> strings. >> >> It should be documented. > > There is this FAQ entry: > > https://www.open-mpi.org/faq/?category=building#installdirs > > -- Reuti > > >> I'd have expected --prefix to have the same >> effect, and for there to be an MCA variable. Would there be some >> problem with either of those? >> ___ >> users mailing list >> users@lists.open-mpi.org >> https://lists.open-mpi.org/mailman/listinfo/users > > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users -- Jeff Squyres jsquy...@cisco.com ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
> Am 09.04.2019 um 14:52 schrieb Dave Love : > > Reuti writes: > >> export OPAL_PREFIX= >> >> to point it to the new location of installation before you start `mpiexec`. > > Thanks; that's now familiar, and I don't know how I missed it with > strings. > > It should be documented. There is this FAQ entry: https://www.open-mpi.org/faq/?category=building#installdirs -- Reuti > I'd have expected --prefix to have the same > effect, and for there to be an MCA variable. Would there be some > problem with either of those? > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
Dave, Can you please post the configure command line of the Open MPI you are trying to relocate ? Cheers, Gilles Dave Love wrote: >Reuti writes: > >> export OPAL_PREFIX= >> >> to point it to the new location of installation before you start `mpiexec`. > >Thanks; that's now familiar, and I don't know how I missed it with >strings. > >It should be documented. I'd have expected --prefix to have the same >effect, and for there to be an MCA variable. Would there be some >problem with either of those? >___ >users mailing list >users@lists.open-mpi.org >https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
Reuti writes: > export OPAL_PREFIX= > > to point it to the new location of installation before you start `mpiexec`. Thanks; that's now familiar, and I don't know how I missed it with strings. It should be documented. I'd have expected --prefix to have the same effect, and for there to be an MCA variable. Would there be some problem with either of those? ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] relocating an installation
Hi, > Am 09.04.2019 um 13:31 schrieb Dave Love : > > Is it possible to use the environment or mpirun flags to run an OMPI > that's been relocated from where it was configured/installed? (Say > you've unpacked a system package that expects to be under /usr and want > to run it from home without containers etc.) I thought that was > possible, but I haven't found a way that works. Using --prefix doesn't > find help files, at least. export OPAL_PREFIX= to point it to the new location of installation before you start `mpiexec`. -- Reuti > ___ > users mailing list > users@lists.open-mpi.org > https://lists.open-mpi.org/mailman/listinfo/users ___ users mailing list users@lists.open-mpi.org https://lists.open-mpi.org/mailman/listinfo/users
Re: [OMPI users] Relocating an Installation
Greetings Allen. This problem has not yet been resolved, but I'm quite sure we have an open ticket about this on our bug tracker. I just replied to Patrick on-list about a related issue (his use of --prefix); I'd have to think about this a bit more, but a solution proposed by one of the other OMPI developers in internal conversations may fix both issues. It only hasn't been coded up because we didn't prioritize it high. So my question to you is -- how high of a priority is this for you? Part of what makes it into each OMPI release is driven by what users want/need, so input like this helps us prioritize the work. Thanks! On Dec 13, 2006, at 10:37 AM, Allen Barnett wrote: There was a thread back in November started by Patrick Jessee about relocating an installation after it was built (the subject was: removing hard-coded paths from OpenMPI shared libraries). I guess I'm in the same boat now. I would like to distribute our OpenMPI-based parallel solver; but I can't really dictate where a user will install our software. Has any one succeeded in building a version of OpenMPI which can be relocated? -- Jeff Squyres Server Virtualization Business Unit Cisco Systems
