Re: [CMake] Multiple occurrences of a library on linux (ldd)
DEBUG=files` to debug linking issues > on linux. It provides more detail on the ldd output, as below: > > 18843: file=libc10.so [0]; needed by > /home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib/libdatareader.so > [0] >(no linking information) > 18843: file=libdatareader.so [0]; needed by > subprojects/Build/datareaders_core_test/standalone_gtests [0] > 18843: file=libdatareader.so [0]; generating link map > (linking information) > (...) > 18843: file=libc10.so [0]; needed by > /home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libcaffe2.so > [0] > 18843: file=libc10.so [0]; generating link map >(linking information) > 18843: file=libcaffe2.so [0]; needed by > subprojects/Build/datareaders_core_test/standalone_gtests [0] > 18843: file=libcaffe2.so [0]; generating link map >(linking information) > > Now I can see that the missing libc10.so is needed by > `libdatareader.so`, which was linked against `standalone_gtests`. > > However, RUNPATH for both `libdatareader.so` and `standalone_gtests` > seems to be correct and point to the same torch/lib folder that has > libc10.so: > libdatareader.so: > RUNPATH=/home/dev/miniconda3/datareaders_py37/build/stage/boost/lib:/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib: > standalone_gtests: > RUNPATH=/home/dev/miniconda3/datareaders_py37/build/stage/boost/lib:/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib:/home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib > > So, libdatareader.so links to libcaffe2.so which links to libc10.so > successfully; standalone_gtests links to libcaffe2.so which links to > libc10.so successully too; however, when standalone_gtests links to > libdatareader.so, there is a transitive linking issue with > libdatareader.so dependency on libc10.so, like the diagram below: > > libdatareader.so ---> libcaffe2.so ---> libc10.so (ok) > > standalone_gtests 0 ---> libcaffe2.so ---> libc10.so (ok) > | > ---> libdatareader.so ---> libc10.so (not found) > > > Thanks again, > Thiago > > > > On Thu, Feb 14, 2019 at 6:13 AM Thompson, KT wrote: > > > > Thiago, > > > > > > > > I haven’t see the double entry pattern that you mention below. However, > > you might want to tell CMake to embed a BUILD_RPATH in your libraries. > > This should get around the issue of manually setting LD_LIBRARY_PATH. > > > > > > > > https://cmake.org/cmake/help/latest/prop_tgt/BUILD_RPATH.html?highlight=rpath > > > > > > > > -kt > > > > > > > > From: CMake On Behalf Of Thiago Crepaldi > > Sent: Thursday, February 14, 2019 12:43 AM > > To: cmake@cmake.org > > Subject: [CMake] Multiple occurrences of a library on linux (ldd) > > > > > > > > Hello all, > > > > > > > > After reading CMake Cookbook I have written my first "complex" CMake build > > script based on the superbuild pattern. I am excited to heave a better > > understanding on CMake, but I definitely will learn much more from > > experience and your kind help. > > > > In summary, the standalone google test application `standalone_gtests` > > publicly links to `libdatareader.so` and to pytorch libraries > > (`libc10.so`,`libcafee2.so`, `libtorch.so`). > > > > `libdatareader.so` also publicly links to pytorch libraries (I have a > > theoretical question on why `standalone_gtests` had to link to pytorch > > libraries if `libdatareader.so` already did, but that can wait). > > > > > > > > Compilation finishes successfully, but when I try to run > > `standalone_gtests`, it aborts because it cant find `libc10.so`. > > > > After executing `ldd standalone_gtests`, the weird result was that there > > were two entries for `libc10.so`. > > > > The first one maps to "not found" while the second had the correct path to > > the library. `libcaffe2.so`, which is also a pytorch library, has a single > > occurrence with full path. > > > > If I add the (...)/pytorch_external/(...) (see ldd output below) path to > > LD_LIBRARY_PATH, then everything works, but I would like to avoid this, if > > possible. > > > > > > > > `ldd ./subprojects/Build/datareaders_core_test/standalone_gtests > > > > libdatareader.so => > > /home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib/libdatareader.so &
Re: [CMake] Multiple occurrences of a library on linux (ldd)
Thanks, Thompson, I will look into BUILD_RPATH and possibly INSTALL_RPATH. I just learned about `export LD_DEBUG=files` to debug linking issues on linux. It provides more detail on the ldd output, as below: 18843: file=libc10.so [0]; needed by /home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib/libdatareader.so [0] (no linking information) 18843: file=libdatareader.so [0]; needed by subprojects/Build/datareaders_core_test/standalone_gtests [0] 18843: file=libdatareader.so [0]; generating link map (linking information) (...) 18843: file=libc10.so [0]; needed by /home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libcaffe2.so [0] 18843: file=libc10.so [0]; generating link map (linking information) 18843: file=libcaffe2.so [0]; needed by subprojects/Build/datareaders_core_test/standalone_gtests [0] 18843: file=libcaffe2.so [0]; generating link map (linking information) Now I can see that the missing libc10.so is needed by `libdatareader.so`, which was linked against `standalone_gtests`. However, RUNPATH for both `libdatareader.so` and `standalone_gtests` seems to be correct and point to the same torch/lib folder that has libc10.so: libdatareader.so: RUNPATH=/home/dev/miniconda3/datareaders_py37/build/stage/boost/lib:/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib: standalone_gtests: RUNPATH=/home/dev/miniconda3/datareaders_py37/build/stage/boost/lib:/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib:/home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib So, libdatareader.so links to libcaffe2.so which links to libc10.so successfully; standalone_gtests links to libcaffe2.so which links to libc10.so successully too; however, when standalone_gtests links to libdatareader.so, there is a transitive linking issue with libdatareader.so dependency on libc10.so, like the diagram below: libdatareader.so ---> libcaffe2.so ---> libc10.so (ok) standalone_gtests 0 ---> libcaffe2.so ---> libc10.so (ok) | ---> libdatareader.so ---> libc10.so (not found) Thanks again, Thiago On Thu, Feb 14, 2019 at 6:13 AM Thompson, KT wrote: > > Thiago, > > > > I haven’t see the double entry pattern that you mention below. However, you > might want to tell CMake to embed a BUILD_RPATH in your libraries. This > should get around the issue of manually setting LD_LIBRARY_PATH. > > > > https://cmake.org/cmake/help/latest/prop_tgt/BUILD_RPATH.html?highlight=rpath > > > > -kt > > > > From: CMake On Behalf Of Thiago Crepaldi > Sent: Thursday, February 14, 2019 12:43 AM > To: cmake@cmake.org > Subject: [CMake] Multiple occurrences of a library on linux (ldd) > > > > Hello all, > > > > After reading CMake Cookbook I have written my first "complex" CMake build > script based on the superbuild pattern. I am excited to heave a better > understanding on CMake, but I definitely will learn much more from experience > and your kind help. > > In summary, the standalone google test application `standalone_gtests` > publicly links to `libdatareader.so` and to pytorch libraries > (`libc10.so`,`libcafee2.so`, `libtorch.so`). > > `libdatareader.so` also publicly links to pytorch libraries (I have a > theoretical question on why `standalone_gtests` had to link to pytorch > libraries if `libdatareader.so` already did, but that can wait). > > > > Compilation finishes successfully, but when I try to run `standalone_gtests`, > it aborts because it cant find `libc10.so`. > > After executing `ldd standalone_gtests`, the weird result was that there were > two entries for `libc10.so`. > > The first one maps to "not found" while the second had the correct path to > the library. `libcaffe2.so`, which is also a pytorch library, has a single > occurrence with full path. > > If I add the (...)/pytorch_external/(...) (see ldd output below) path to > LD_LIBRARY_PATH, then everything works, but I would like to avoid this, if > possible. > > > > `ldd ./subprojects/Build/datareaders_core_test/standalone_gtests > > libdatareader.so => > /home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib/libdatareader.so > > libcaffe2.so => > /home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libcaffe2.so > > libc10.so => not found > > libc10.so => > /home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libc10.so > > (...)` > > > > Have anyone seen multiple entries for the same library on ldd before? Why is > that? Is it because `standalone_gte
Re: [CMake] Multiple occurrences of a library on linux (ldd)
Thiago,
I haven’t see the double entry pattern that you mention below. However, you
might want to tell CMake to embed a BUILD_RPATH in your libraries. This should
get around the issue of manually setting LD_LIBRARY_PATH.
https://cmake.org/cmake/help/latest/prop_tgt/BUILD_RPATH.html?highlight=rpath
-kt
From: CMake On Behalf Of Thiago Crepaldi
Sent: Thursday, February 14, 2019 12:43 AM
To: cmake@cmake.org
Subject: [CMake] Multiple occurrences of a library on linux (ldd)
Hello all,
After reading CMake Cookbook I have written my first "complex" CMake build
script based on the superbuild pattern. I am excited to heave a better
understanding on CMake, but I definitely will learn much more from experience
and your kind help.
In summary, the standalone google test application `standalone_gtests` publicly
links to `libdatareader.so` and to pytorch libraries
(`libc10.so`,`libcafee2.so`, `libtorch.so`).
`libdatareader.so` also publicly links to pytorch libraries (I have a
theoretical question on why `standalone_gtests` had to link to pytorch
libraries if `libdatareader.so` already did, but that can wait).
Compilation finishes successfully, but when I try to run `standalone_gtests`,
it aborts because it cant find `libc10.so`.
After executing `ldd standalone_gtests`, the weird result was that there were
two entries for `libc10.so`.
The first one maps to "not found" while the second had the correct path to the
library. `libcaffe2.so`, which is also a pytorch library, has a single
occurrence with full path.
If I add the (...)/pytorch_external/(...) (see ldd output below) path to
LD_LIBRARY_PATH, then everything works, but I would like to avoid this, if
possible.
`ldd ./subprojects/Build/datareaders_core_test/standalone_gtests
libdatareader.so =>
/home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib/libdatareader.so
libcaffe2.so =>
/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libcaffe2.so
libc10.so => not found
libc10.so =>
/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libc10.so
(...)`
Have anyone seen multiple entries for the same library on ldd before? Why is
that? Is it because `standalone_gtests` links to libc10.so and to
`libdatareader.so`, which in turn also links to `libc10.so`?
Both CMakeLists.txt (libdatareader and standalone_gtests) succeeds at
find_package(Torch REQUIRED QUIET) commands (${TORCH_LIBRARY} returns the
correct path).
I run the same build on my Mac and everything works fine, so that is confined
to linux environment. I have destroyed my conda environment and performed
multiple clean builds in the process and no luck :(
Hoping this was some sort of ldconfig issue, I tried `sudo ldconfig` and `sudo
rm /etc/ld.so.cache`, but that doesn't fix it.
Any ideas?
best regards,
Thiago
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
[CMake] Multiple occurrences of a library on linux (ldd)
Hello all,
After reading CMake Cookbook I have written my first "complex" CMake build
script based on the superbuild pattern. I am excited to heave a better
understanding on CMake, but I definitely will learn much more from
experience and your kind help.
In summary, the standalone google test application `standalone_gtests`
publicly links to `libdatareader.so` and to pytorch libraries
(`libc10.so`,`libcafee2.so`, `libtorch.so`).
`libdatareader.so` also publicly links to pytorch libraries (I have a
theoretical question on why `standalone_gtests` had to link to pytorch
libraries if `libdatareader.so` already did, but that can wait).
Compilation finishes successfully, but when I try to run
`standalone_gtests`, it aborts because it cant find `libc10.so`.
After executing `ldd standalone_gtests`, the weird result was that there
were two entries for `libc10.so`.
The first one maps to "not found" while the second had the correct path to
the library. `libcaffe2.so`, which is also a pytorch library, has a single
occurrence with full path.
If I add the (...)/pytorch_external/(...) (see ldd output below) path to
LD_LIBRARY_PATH, then everything works, but I would like to avoid this, if
possible.
`ldd ./subprojects/Build/datareaders_core_test/standalone_gtests
libdatareader.so =>
/home/dev/miniconda3/datareaders_py37/build/stage/datareader/lib/libdatareader.so
libcaffe2.so =>
/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libcaffe2.so
libc10.so => not found
libc10.so =>
/home/dev/miniconda3/datareaders_py37/build/subprojects/Source/pytorch_external/torch/lib/libc10.so
(...)`
Have anyone seen multiple entries for the same library on ldd before? Why
is that? Is it because `standalone_gtests` links to libc10.so and to
`libdatareader.so`, which in turn also links to `libc10.so`?
Both CMakeLists.txt (libdatareader and standalone_gtests) succeeds at
find_package(Torch REQUIRED QUIET) commands (${TORCH_LIBRARY} returns the
correct path).
I run the same build on my Mac and everything works fine, so that is
confined to linux environment. I have destroyed my conda environment and
performed multiple clean builds in the process and no luck :(
Hoping this was some sort of ldconfig issue, I tried `sudo ldconfig` and
`sudo rm /etc/ld.so.cache`, but that doesn't fix it.
Any ideas?
best regards,
Thiago
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake
