Maybe have an opt-in (say —with-rpath)? -Jojy
> On Feb 12, 2016, at 2:19 PM, Kevin Klues <[email protected]> wrote: > > To be clear, I'm actually a bit torn both ways on this. > > On the one hand, including the rpath makes it easy for those who don't > know anything about LD_LIBRARY_PATH, the ldcache, etc. to simply pass > their paths to their external dependencies at configure time and then > run their binaries without further effort. > > On the other hand, maybe they should be cognizant of the fact that > something is going on under the hood to actually allow their binaries > to link properly (i.e. I can imagine a situation where someone builds, > runs, and tests everything locally, and then is confused as to why it > nothing works once deployed). > > In my previous email, I argue that we should include the rpath by > default, and can strip it later if we don't want it for some reason > (i.e. when bundling into debs/rpms). Conversely, we could leave it > out by default and only set it as a post-processing step in situations > where we actually care about it. > > I'm curious what other people's thoughts are. > > > On Fri, Feb 12, 2016 at 1:47 PM, Kevin Klues <[email protected]> wrote: >> Hi all, >> >> A discussion came up recently around including rpaths in our mesos >> binaries to help resolve any shared library dependencies that don't >> exist in standard library paths (e.g. /lib, /usr/local/lib, etc.). >> >> By default, there are no shared library dependencies that exist in >> non-standard paths, because we bundle all of these dependencies into >> the mesos source and statically link them into our executables (e.g. >> glog, zookeeper, etc.) >> >> However, if you configure mesos with e.g. >> >> ../configure ----disable-bundled >> >> or the more selective >> >> ../configure --with-glog[=DIR] --with-zookeeper[=DIR] ... >> >> then mesos will be built with an external shared library dependency >> (e.g. glog and zookeeper in this case). >> >> The build system is smart enough to set up LDFLAGS so we can link >> against whatever external libraries are passed in via the --with-* >> flags. >> >> However, when we go to run the binaries that are produced (e.g. >> mesos-master, mesos-slave, mesos-test, etc.), we have to prefix them >> with an LD_LIBRARY_PATH pointing to the location of the shared >> libraires from these external dependencies, e.g. >> >> LD_LIBRARY_PATH="/glog/lib:/zookeeper/lib" ./mesos-master >> >> It would be nice if we didn't have to explicitly set the >> LD_LIBRARY_PATH to launch these binaries when linking against any >> external shared library dependencies. >> >> One way around this would be to make sure that all external library >> dependencies were stored in standard search paths for the dynamic >> linker. This is typically what happens if you install these >> dependencies via a standard package manager (e.g. apt-get, yum, etc.). >> Sometimes this is undesirable (or impossible) though, especially if >> the external dependencies do not exist as packages or follow a >> non-standard directory hierarchy in terms of where it places its >> include files, libraries, etc. >> >> Another option is to install the paths to these external libraries >> into the ldcache (e.g. via /etc/ld.so.conf on linux) so that the >> dynamic linker will search them at runtime. This is also unfeasible >> at times and has the added disadvantage that these library paths will >> now be searched for *all* binaries that get executed (not just the >> ones we currently care about). >> >> The final option (and the one I'm proposing here) is to set the >> 'rpath' of the binary to point to the location of the external shared >> library on the build machine. The rpath is embedded into the binary >> at link time and is used to give the linker an extra set of paths to >> search for shared libraries at runtime. This obvious advantage here >> is that setting rpath allows us to run our binaries without requiring >> LD_LIBRARY_PATH or any of the other methods mentioned above to tell >> the linker where to find our shared libraries. However, it has the >> disadvantage of baking a path into the binary that may only exist on >> the specific machine the binary was built on. >> >> That said, the standard search order used by the dynamic linker to >> find shared libraries is: >> >> 1) LD_LIBRARY_PATH >> 2) rpath >> 4) the ldcache (/etc/ld.so.conf on linux) >> 3) default paths (e.g. /lib, /usr/local/lib) >> >> Meaning that we could always overwrite the rpath using LD_LIBARY_PATH >> if we wanted to. Moreover, we could even change the rpath at the time >> of deployment (e.g. via chrpath on linux). This may be desirable if >> the shared libraries are installed at different locations on the >> deployment machine. >> >> If there are no objections, I therefore propose we modify the >> following files to add rpaths to all external dependencies set via >> --with-* flags: >> >> ./configure.ac >> ./3rdparty/libprocess/3rdparty/stout/configure.ac >> ./3rdparty/libprocess/configure.ac >> >> The pattern would change from: >> >> CPPFLAGS="-I${with_thing}/include $CPPFLAGS" >> LDFLAGS="-L${with_thing}/lib $LDFLAGS" >> >> to include an additional line with: >> >> LDFLAGS="-Wl,-rpath,${with_thing}/lib $LDFLAGS" >> >> I know there has some hesitation with this in the past (especially >> when it comes to producing rpms or debs, where baking in an rpath >> seems really strange), but I'm hoping people will agree that it's >> useful enough that it makes sense to include the rpaths as the default >> case. We can always run a post-processing step to strip them in cases >> where they are undesirable. >> >> Thanks! >> >> -- >> ~Kevin > > > > -- > ~Kevin
