Reuti -- thanks for the comments+fix about missing "-Wl," (oops!). In addition to yours, some more came in on https://github.com/open-mpi/ompi/pull/10624 after it was merged. I'll make a follow-on PR with these suggestions.
-- Jeff Squyres jsquy...@cisco.com ________________________________ From: Reuti Sent: Tuesday, August 9, 2022 12:03 PM To: Open MPI Users Cc: Jeff Squyres (jsquyres); zuelc...@staff.uni-marburg.de Subject: Re: [OMPI users] RUNPATH vs. RPATH Hi Jeff, > Am 09.08.2022 um 16:17 schrieb Jeff Squyres (jsquyres) via users > <users@lists.open-mpi.org>: > > Just to follow up on this thread... > > Reuti: I merged the PR on to the main docs branch. They're now live -- we > changed the text: > • here: > https://docs.open-mpi.org/en/main/installing-open-mpi/configure-cli-options/installation.html On this page I read: Using --disable-wrapper-rpath will disable both “runpath” and “rpath” behavior in the wrapper compilers. I would phrase it: Using --disable-wrapper-rpath in addition will disable both “runpath” and “rpath” behavior in the wrapper compilers. (otherwise I get a "configure: error: --enable-wrapper-runpath cannot be selected with --disable-wrapper-rpath") > • and here: > https://docs.open-mpi.org/en/main/installing-open-mpi/configure-cli-options/rpath-and-runpath.html The last command reads `shell$ ./configure LDFLAGS=--enable-new-dtags ...`. But the LDFLAGS will be given to the compiler wrapper, hence it seems to need -Wl,--enable-new-dtags what I used initially to avoid: configure:6591: checking whether the C compiler works configure:6613: gcc --enable-new-dtags conftest.c >&5 cc1: error: unknown pass new-dtags specified in -fenable > Here's the corresponding PR to update the v5.0.x docs: > https://github.com/open-mpi/ompi/pull/10640 > > Specifically, the answer to your original question is twofold: > • It's complicated. 🙂 > • It looks like you did the Right Thing for your environment, but you > might want to check the output of "readelf -d ..." to be sure. > Does that additional text help explain things? Yes, thx a lot for the clarification and update of the documentation. -- Reuti > -- > Jeff Squyres > jsquy...@cisco.com > From: Jeff Squyres (jsquyres) <jsquy...@cisco.com> > Sent: Saturday, August 6, 2022 9:36 AM > To: Open MPI Users <users@lists.open-mpi.org> > Subject: Re: [OMPI users] RUNPATH vs. RPATH > > Reuti -- > > See my disclaimers on other posts about apologies for taking so long to reply! > > This code was written forever ago; I had to dig through it a bit, read the > comments and commit messages, and try to remember why it was done this way. > What I thought would be a 5-minute search turned into a few hours of digging > through code, multiple conversations with Brian, and one pull request (so > far). We don't have a definitive answer yet, but we think we're getting > closer. > > The short version is that what you did appears to be correct: > > ./configure LDFLAGS=-Wl,--enable-new-dtags ... > > The longer answer is that whenever you think you understand the shared > library and run-time linkers, you inevitably find out that you don't. The > complicated cases come from the fact that the handling of rpath and runpath > can be different on different platforms, and there are subtle differences in > their behavior (beyond the initial "search before or after LD_LIBRARY_PATH, > such as the handling of primary and secondary/transitive dependencies). > > The pull request I have so far is https://github.com/open-mpi/ompi/pull/10624 > (rendered here: > https://ompi--10624.org.readthedocs.build/en/10624/installing-open-mpi/configure-cli-options/installation.html). > We're not 100% confident in that text yet, but I think we're close to at > least documenting what the current behavior is. Once we nail that down, we > can talk about whether we want to change that behavior. > > > ________________________________________ > From: users <users-boun...@lists.open-mpi.org> on behalf of Reuti via users > <users@lists.open-mpi.org> > Sent: Friday, July 22, 2022 9:48 AM > To: Open MPI Users > Cc: Reuti; zuelc...@staff.uni-marburg.de > Subject: [OMPI users] RUNPATH vs. RPATH > > Hi, > > using Open MPI 4.1.4 > > $ mpicc --show … > > tells me, that the command line contains "… -Wl,--enable-new-dtags …" so that > even older linkers will include RUNPATH instead of RPATH in the created > dynamic binary. On the other hand, Open MPI itself doesn't use this option > for its own libraries: > > ./liboshmem.so.40.30.2 > ./libmpi_mpifh.so.40.30.0 > ./libmpi.so.40.30.4 > ./libmpi_usempi_ignore_tkr.so.40.30.0 > ./libopen-rte.so.40.30.2 > > Is this intended? > > Setting LD_LIBRARY_PATH will instruct the created binary to look for > libraries first in that location and resolve it, but the loaded library in > turn will then use RPATH inside itself first to load additional libraries. > > (I compile Open MPI in my home directory and move it then to the final > destination for the group; setting OPAL_PREFIX of course. I see a mix of > library locations when I run the created binary on my own with `ldd`.) > > Looks like I can get the intended behavior while configuring Open MPI on this > (older) system: > > $ ./configure … LDFLAGS=-Wl,--enable-new-dtags > > -- Reuti
