Our compiler activation scripts (highlighting the bit of most interest
to you I hope) are here:
https://github.com/AnacondaRecipes/aggregate/blob/master/ctng-compilers-activation-feedstock/recipe/activate-gcc.sh#L84-L101
On Wed, Aug 15, 2018 at 11:44 AM, Ray Donnelly <mingw.andr...@gmail.com> wrote:
> Hi Sebastián,
>
> Without having time to properly go through this, I feel I should
> correct some technical inaccuracies, but *all* of your issues can be
> sidestepped by using conda-build. Installation and RPATHs are always
> very tricky for projects to get right so we side step any issues here
> by running post-build fixups on these things. We ensure RPATHs have
> the opportunity to add our own, uniquify them (I think!), and make
> them fully relative on macOS and Linux.
>
> WRT RPATHs being added or not, we set LDFLAGs for RPATHs the linker
> activation script *when run under conda-build*. If you want to fake
> that so you get the exact same flags that are used to compile our
> packages do: CONDA_BUILD=1 conda activate envname. If you don't set
> CONDA_BUILD=1 then it'll set all flags *except* -I${PREFIX}/include,
> -L${PREFIX}/lib and -Wl,-rpath,${PREFIX}/lib.
>
> If you want to use our compilers in their completely 'raw' form (of
> course you could unset LDFLAGS, CFLAGS, CPPFLAGS too), then install
> e.g. gcc_impl_linux-64 instead of gcc_linux-64 which really are
> helpers for interfacing the raw compilers with conda-build (and
> setting good default compilation flags for security and performance
> reasons).
>
> You mentioned using [DY]LD_LIBRARY_PATH, I would caution no one to do
> this ever outside of development workflows. FWIW, we run into lots of
> trouble with LD_LIBRARY_PATH and I am *firmly* in the camp that
> LD_LIBRARY_PATH is harmful (i.e. never use it systematically) and I am
> utterly convinced that DT_RUNPATH is a huge mistake. We actually
> consider detection of DT_RUNPATH in any of our DSOs or exes an error
> condition and only allow DT_RPATH. We just had so many bugs due to the
> wrong libs being used by end users due to this, for example there is
> no way a Qt application linked against our Qt libs is going to be
> happy with some random system Qt library that's been interposed.
>
> For macOS, you should also set CONDA_BUILD_SYSROOT=<somewhere that
> macOS 10.9, 10.10 or 10.11 SDK> (depending on the macOS version you
> want to target). These old SDKs can be gotten from old Xcode builds
> and also on github. Unfortunately the compilers are not compatible
> with the new .tbd file format used in newer SDKs and by the newest
> Apple system linker and the source to enable that has not been
> released yet (there have been no new source drops for Apple's ld64 in
> a while).
>
> But please, just use conda-build! While we try to keep them working in
> as many situations as we can, these tools are primarily focussed
> around conda-build and the technical decisions we make with respect to
> them will be from that perspective.
>
> Hope this helps some?
>
> Fri, Aug 10, 2018 at 10:48 PM, Sebastián Mancilla <smanc...@jlab.org> wrote:
>> I am trying to use Conda as a package manager for isolated C++ development
>> environments. But unfortunately, when using CMake with the Anaconda-provided
>> compilers [1] (which are used to compile the binary packages in the Anaconda
>> repositories), things do not work as expected.
>>
>> I have a small test case available here [2], with an executable calling a
>> shared library and a third-party dependency installed with Conda.
>>
>> [1]:
>> https://conda.io/docs/user-guide/tasks/build-packages/compiler-tools.html
>> [2]: https://gist.github.com/smancill/b28ca07ac11fdf285b4d559545a1630b
>>
>> --------------------------------------------------
>>
>> First, when using the system compiler, all works fine (but I am not sure of
>> the
>> binary compatibility with the Conda packages, that's why I want to use the
>> Anaconda compilers):
>>
>> # create the environment and install XercesC
>> $ conda create -n test-system xerces-c
>> ...
>> environment location:
>> /Users/smancill/.local/share/miniconda3/envs/test-system
>> ...
>> The following NEW packages will be INSTALLED:
>>
>> icu: 58.2-h4b95b61_1
>> libcxx: 4.0.1-h579ed51_0
>> libcxxabi: 4.0.1-hebd6815_0
>> xerces-c: 3.2.1-h44e365a_0
>> ...
>>
>> # activate the environment
>> $ conda activate test-system
>>
>> $ mkdir build-osx-system
>> $ cd build-osx-system
>> $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
>> -DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
>> -- The CXX compiler identification is AppleClang 9.0.0.9000039
>> -- Check for working CXX compiler: /usr/bin/c++
>> -- Check for working CXX compiler: /usr/bin/c++ -- works
>> ...
>> -- Found XercesC:
>> /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib
>> (found version "3.2.1")
>> -- Configuring done
>> -- Generating done
>> -- Build files have been written to:
>> /Users/smancill/src/conda-test/build-osx-system
>>
>> $ make -j1 VERBOSE=1
>> ...
>> [100%] Linking CXX executable bar
>> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
>> --verbose=1
>> /usr/bin/c++ -isysroot
>> /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
>> -mmacosx-version-min=10.12 -Wl,-search_paths_first
>> -Wl,-headerpad_max_install_names CMakeFiles/bar.dir/bar.cpp.o -o bar
>> -Wl,-rpath,/Users/smancill/src/conda-test/build-osx-system
>> -Wl,-rpath,/Users/smancill/.local/share/miniconda3/envs/test-system/lib
>> libfoo.dylib
>> /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libxerces-c.dylib
>> ...
>>
>> The build directory (~/src/conda-test/build-osx-system) and the conda
>> environment lib directory (~/.local/share/miniconda3/envs/test-system/lib)
>> are correctly added to the RPATH in the build tree by the link command:
>>
>> $ ./bar
>> Hello, world!
>>
>> $ otool -L ./bar
>> ./bar:
>> @rpath/libfoo.dylib (compatibility version 1.0.0, current version
>> 0.0.0)
>> @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
>> version 400.9.0)
>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
>> version 1252.0.0)
>>
>> $ otool -l ./bar | grep -A2 LC_RPATH
>> cmd LC_RPATH
>> cmdsize 56
>> path /Users/smancill/src/conda-test/build-osx-system (offset 12)
>> --
>> cmd LC_RPATH
>> cmdsize 80
>> path
>> /Users/smancill/.local/share/miniconda3/envs/test-system/lib (offset 12)
>>
>> If I install the binary, it fails because I haven't configured CMake to set
>> the install RPATH:
>>
>> $ make install
>> ...
>> Install the project...
>> -- Install configuration: ""
>> -- Installing:
>> /Users/smancill/.local/share/miniconda3/envs/test-system/lib/libfoo.dylib
>> -- Installing:
>> /Users/smancill/.local/share/miniconda3/envs/test-system/include/foo.hpp
>> -- Installing:
>> /Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar
>>
>> $ bar
>> dyld: Library not loaded: @rpath/libfoo.dylib
>> Referenced from:
>> /Users/smancill/.local/share/miniconda4/envs/test-system/bin/bar
>> Reason: image not found
>> [1] 84611 abort bar
>>
>> $ otool -L $CONDA_PREFIX/bin/bar
>> /Users/smancill/.local/share/miniconda3/envs/test-system/bin/bar:
>> @rpath/libfoo.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>> @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>> /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current
>> version 400.9.0)
>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
>> version 1252.0.0)
>>
>> $ otool -l $CONDA_PREFIX/bin/bar | grep -A2 LC_RPATH
>> # empty
>>
>> # deactivate the environment to start again
>> $ conda deactivate
>>
>> The same can be observed on Linux. Everything works as it should.
>>
>> --------------------------------------------------
>>
>> If I try to use Anaconda compilers on macOS, the build RPATH is not set
>> properly anymore:
>>
>> # create the environment and install the Anaconda compiler for macOS,
>> and XercesC
>> $ conda create -n test-conda clangxx_osx-64 xerces-c
>> ...
>> environment location:
>> /Users/smancill/.local/share/miniconda3/envs/test-conda
>> ...
>> The following NEW packages will be INSTALLED:
>>
>> cctools: 895-h7512d6f_0
>> clang: 4.0.1-h662ec87_0
>> clang_osx-64: 4.0.1-h1ce6c1d_11
>> clangxx: 4.0.1-hc9b4283_0
>> clangxx_osx-64: 4.0.1-h22b1bf0_11
>> compiler-rt: 4.0.1-h5487866_0
>> icu: 58.2-h4b95b61_1
>> ld64: 274.2-h7c2db76_0
>> libcxx: 4.0.1-h579ed51_0
>> libcxxabi: 4.0.1-hebd6815_0
>> llvm: 4.0.1-hc748206_0
>> llvm-lto-tapi: 4.0.1-h6701bc3_0
>> xerces-c: 3.2.1-h44e365a_0
>> ...
>>
>> # activate the environment (which sets the variables to use the Anaconda
>> compiler)
>> $ conda activate test-conda
>>
>> $ mkdir build-osx-conda
>> $ cd build-osx-conda
>> $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
>> -DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
>> -- The CXX compiler identification is Clang 4.0.1
>> -- Check for working CXX compiler:
>> /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
>> -- Check for working CXX compiler:
>> /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
>> -- works
>> ...
>> -- Found XercesC:
>> /Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib
>> (found version "3.2.1")
>> -- Configuring done
>> -- Generating done
>> -- Build files have been written to:
>> /Users/smancill/src/conda-test/build-osx-conda
>>
>> $ make -j1 VERBOSE=1
>> ...
>> [100%] Linking CXX executable bar
>> /usr/local/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
>> --verbose=1
>>
>> /Users/smancill/.local/share/miniconda3/envs/test-conda/bin/x86_64-apple-darwin13.4.0-clang++
>> -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE
>> -fstack-protector-strong -O2 -pipe -stdlib=libc++
>> -fvisibility-inlines-hidden -std=c++14 -fmessage-length=0 -isysroot
>> /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
>> -mmacosx-version-min=10.12 -Wl,-search_paths_first
>> -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names
>> -Wl,-dead_strip_dylibs CMakeFiles/bar.dir/bar.cpp.o -o bar
>> -Wl,-rpath,/Users/smancill/src/conda-test/build-osx-conda libfoo.dylib
>> /Users/smancill/.local/share/miniconda3/envs/test-conda/lib/libxerces-c.dylib
>> ...
>>
>> You can see that the environment lib path is not added to the RPATH by the
>> link command,
>> which in turns result in the executable not running from the build tree
>> anymore:
>>
>> $ ./bar
>> dyld: Library not loaded: @rpath/libxerces-c-3.2.dylib
>> Referenced from: /Users/smancill/src/conda-test/build-osx-conda/./bar
>> Reason: image not found
>> [1] 89350 abort ./bar
>>
>> $ otool -L ./bar
>> ./bar:
>> @rpath/libfoo.dylib (compatibility version 0.0.0, current version
>> 0.0.0)
>> @rpath/libxerces-c-3.2.dylib (compatibility version 0.0.0, current
>> version 0.0.0)
>> @rpath/libc++.1.dylib (compatibility version 1.0.0, current version
>> 1.0.0)
>> /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current
>> version 1252.0.0)
>>
>> $ otool -l ./bar | grep -A2 LC_RPATH
>> cmd LC_RPATH
>> cmdsize 56
>> path /Users/smancill/src/conda-test/build-osx-conda (offset 12)
>>
>> # deactivate the environment
>> $ conda deactivate
>>
>> --------------------------------------------------
>>
>> If I try the Anaconda compilers on Linux, there are strange results too:
>>
>> # create the environment and install the Anaconda compiler for Linux,
>> and XercesC
>> $ conda create -n test-conda gxx_linux-64 xerces-c
>> ...
>> environment location: /home/vagrant/miniconda3/envs/test-conda
>> ...
>> The following NEW packages will be INSTALLED:
>>
>> binutils_impl_linux-64: 2.28.1-had2808c_3
>> binutils_linux-64: 7.2.0-had2808c_27
>> gcc_impl_linux-64: 7.2.0-habb00fd_3
>> gcc_linux-64: 7.2.0-h550dcbe_27
>> gxx_impl_linux-64: 7.2.0-hdf63c60_3
>> gxx_linux-64: 7.2.0-h550dcbe_27
>> icu: 58.2-h9c2bf20_1
>> libgcc-ng: 7.2.0-hdf63c60_3
>> libstdcxx-ng: 7.2.0-hdf63c60_3
>> xerces-c: 3.2.1-hac72e42_0
>>
>> # activate the environment (which sets the variables to use the Anaconda
>> compiler)
>> $ conda activate test-conda
>>
>> $ mkdir build-linux-conda
>> $ cd build-linux-conda
>> $ cmake -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX
>> -DCMAKE_PREFIX_PATH=$CONDA_PREFIX ..
>> -- The CXX compiler identification is GNU 7.2.0
>> -- Check for working CXX compiler:
>> /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
>> -- Check for working CXX compiler:
>> /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
>> -- works
>> ...
>> -- Found XercesC:
>> /home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so (found version
>> "3.2.1")
>> -- Configuring done
>> -- Generating done
>> -- Build files have been written to:
>> /vagrant/conda-test/build-linux-conda
>>
>> $ make -j1 VERBOSE=1
>> ...
>> [100%] Linking CXX executable bar
>> /usr/bin/cmake -E cmake_link_script CMakeFiles/bar.dir/link.txt
>> --verbose=1
>>
>> /home/vagrant/miniconda3/envs/test-conda/bin/x86_64-conda_cos6-linux-gnu-c++
>> -fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona
>> -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2
>> -pipe -Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now
>> CMakeFiles/bar.dir/bar.cpp.o -o bar libfoo.so
>> /home/vagrant/miniconda3/envs/test-conda/lib/libxerces-c.so
>> -Wl,-rpath,/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:
>> ...
>>
>> You can see that the environment lib path is added to the RPATH by the link
>> command (unlike macOS):
>>
>> $ ./bar
>> Hello World!
>>
>> But when inspecting the dependencies, the environment lib path appears
>> twice:
>>
>> $ readelf -d ./bar
>> ...
>> 0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
>> 0x0000000000000001 (NEEDED) Shared library:
>> [libxerces-c-3.2.so]
>> 0x0000000000000001 (NEEDED) Shared library:
>> [libstdc++.so.6]
>> 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
>> 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
>> 0x000000000000000f (RPATH) Library rpath:
>> [/home/vagrant/miniconda3/envs/test-conda/lib:/vagrant/conda-test/build-linux-conda:/home/vagrant/miniconda3/envs/test-conda/lib:]
>> ...
>>
>> Which is wrong. Now the build tree binary will pick first any old version of
>> the foo library installed in the environment instead of the version in the
>> build tree.
>>
>> It seems that the Anaconda toolchain is setting the environment lib path
>> into
>> the RPATH by its own. It is also set when installing the binaries, even when
>> CMake is not configured to set the install RPATH:
>>
>> $ make install
>> ...
>> Install the project...
>> -- Install configuration: ""
>> -- Installing: /home/vagrant/miniconda3/envs/test-conda/lib/libfoo.so
>> -- Installing: /home/vagrant/miniconda3/envs/test-conda/include/foo.hpp
>> -- Installing: /home/vagrant/miniconda3/envs/test-conda/bin/bar
>> -- Set runtime path of
>> "/home/vagrant/miniconda3/envs/test-conda/bin/bar" to ""
>>
>> $ bar
>> Hello World!
>>
>> $ readelf -d $CONDA_PREFIX/bin/bar
>> ...
>> 0x0000000000000001 (NEEDED) Shared library: [libfoo.so]
>> 0x0000000000000001 (NEEDED) Shared library:
>> [libxerces-c-3.2.so]
>> 0x0000000000000001 (NEEDED) Shared library:
>> [libstdc++.so.6]
>> 0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
>> 0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
>> 0x000000000000000f (RPATH) Library rpath:
>> [/home/vagrant/miniconda3/envs/test-conda/lib]
>> ...
>>
>> # deactivate the environment
>> $ conda deactivate
>>
>> --------------------------------------------------
>>
>> TL;DR I cannot get CMake and the Anaconda compilers and packages working
>> correctly.
>>
>> - On macOS, the Conda environment library path is not added to the build
>> RPATH.
>> - On Linux, the Conda environment library path is always added to the RPATH
>> (in both build and install) independently of CMake.
>>
>> Any advice or workarounds?
>>
>> --
>> Sebastian Mancilla
>>
>> --
>>
>> Powered by www.kitware.com
>>
>> Please keep messages on-topic and check the CMake FAQ at:
>> http://www.cmake.org/Wiki/CMake_FAQ
>>
>> Kitware offers various services to support the CMake community. For more
>> information on each offering, please visit:
>>
>> CMake Support: http://cmake.org/cmake/help/support.html
>> CMake Consulting: http://cmake.org/cmake/help/consulting.html
>> CMake Training Courses: http://cmake.org/cmake/help/training.html
>>
>> Visit other Kitware open-source projects at
>> http://www.kitware.com/opensource/opensource.html
>>
>> Follow this link to subscribe/unsubscribe:
>> https://cmake.org/mailman/listinfo/cmake
>>
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
