On 6/26/15, 10:09 AM, "Ward Poelmans" <[email protected]> wrote:
>On Fri, Jun 26, 2015 at 7:04 PM, Todd Gamblin <[email protected]> wrote: >> In Spack, the compilers don't actually have to be loaded, so your new >> compilers stay out of the way of the system software. The stuff >>compiled >> with Spack gets RPATH'd, so it knows how to find its libraries -- in the >> gcc build we actually modify gcc's spec to put an RPATH in to each >> compiler's runtime. That's for compilers Spack actually built -- for >> system compilers, I'm still working on a way to make the Spack build >> automatically RPATH in compiler runtime libraries. It *is* possible to >>do >> this. CMake actually detects what it calls "implicit link libraries" >>for >> each compiler. Most compilers can be coaxed into telling you what these >> are. > >And what about the case where the user wants a compiler loaded to >compile some software himself? Is that handled by rpath's? No, but there are still modules (and dotkits, which no one besides us uses) generated if you want to use the compilers you built within your own environment. You can either use the module Spack generates, or you can just go through spack and use the same syntax, e.g.: spack load [email protected] That queries for installed gcc packages at version 4.8.2 and loads the module if it finds it. For stuff Spack doesn't build, like the intel compilers, we actually install them and change the compiler configuration so that binaries built by a particular compiler can find its runtime libraries. We do similar stuff with MPI compiler wrappers. RPATH is just a field that gets embedded into ELF and mach-o binaries (not sure whether COFF actually supports it). Linking with RPATH doesn't do *anything* to the user's environment. It just ensures that the dynamic loader knows where to find a binary's dependencies when it is run. No LD_LIBRARY_PATH needed. The end result is that users don't have to remember how a binary was built in order to run it. This gets important if, say, you have two applications that the developers like to build with very different compiler toolchains. If you don't RPATH the compiler runtime and the dependencies, you're asking for trouble because you have to load a bunch of modules, run the first app, unload them, and run the second. You could obviously also run them in wrapper scripts that set the environment for the app, but who wants to remember all that for all the dependencies? Our users tend to have scripts that use absolute paths to a particular executable. They seldom remember all the modules they had to load to get a working environment for that executable. The one disadvantage you might find with RPATH is that you can't swap versions of dependencies by setting LD_LIBRARY_PATH. You're pretty much tied to the one you built with -- because RPATH supersedes LD_LIBRARY_PATH. There is RUNPATH for that -- it's basically RPATH with precedence below that of LD_LIBRARY_PATH. We could modify spack so that it just sets RUNPATH flags instead of RPATH ones, but RUNPATH isn't as universally supported, and no one's asked for it yet. If you did this, you could get the RUNPATH settings by just unsetting LD_LIBRARY_PATH, and you could experiment with new versions of dependencies by setting it. In the HPC world, I find that to be a pretty rare case. MPI doesn't even have an ABI, so you're asking for trouble by trying to swap it in. You need something like the hierarchical modules to do that -- where you swap in/out entire trees. -Todd > >Ward
