Re: [CMake] linking static libs into shared libs on Linux
On 07/08, Dan Kegel wrote: So maybe the yasm source needs to have a keyword or two added to select PIC addressing mode? Random googling finds suggestions at http://cvs.tortall.net/pipermail/bug-yasm/2011-October/86.html Thanks for all the help. Upon further debugging with 'make VERBOSE=1' I discovered that when multilib was adding its extra link path it was overwriting the linux link flags instead of appending to them. There was a one-line fix for it. Cheers -- Steve Borho -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
On 07/07, Dan Kegel wrote: On Tue, Jul 7, 2015 at 2:25 PM, Steve Borho st...@borho.org wrote: We're already adding -fPIC to the compile flags for the two object libraries. This way one set of objects can be used to output the shared library and the static library. So the C++ files are already compiled with -fPIC. It is only a problem when we need to link a static library from one build into the shared library of another build. The static library itself no longer supports PIC. Really? I didn't think ar was in the habit of modifying machine code like that. At this point maybe the thing to do is make a toy example that reproduces the problem. (Ideally something truly stipped down, like https://github.com/dankegel/winezeug/tree/master/cmake_examples/ex7 ) This rings a bell for me. We encountered this link failure once before. A little back history: Our project has two main source folders, common/ and encoder/, each with its own CMakeLists file. Originally we built each folder into a static library and then the root level CMakeLists file linked them into a shared library, static library, or executable. This broke when we introduced yasm-compiled assembly objects into the common/ folder. We begain getting these same exact PIC errors when trying to link the static common.lib and encoder.lib into libx265.so. And this is when we were forced to use cmake's object libraries. Now common/ and encoder/ generate object libraries and the root level CMakeLists links those objects into the final targets and this seemed to work ok. So the yasm objects are somehow the root cause. -- Steve Borho -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
So maybe the yasm source needs to have a keyword or two added to select PIC addressing mode? Random googling finds suggestions at http://cvs.tortall.net/pipermail/bug-yasm/2011-October/86.html On Wed, Jul 8, 2015 at 10:11 AM, Steve Borho st...@borho.org wrote: On 07/07, Dan Kegel wrote: On Tue, Jul 7, 2015 at 2:25 PM, Steve Borho st...@borho.org wrote: We're already adding -fPIC to the compile flags for the two object libraries. This way one set of objects can be used to output the shared library and the static library. So the C++ files are already compiled with -fPIC. It is only a problem when we need to link a static library from one build into the shared library of another build. The static library itself no longer supports PIC. Really? I didn't think ar was in the habit of modifying machine code like that. At this point maybe the thing to do is make a toy example that reproduces the problem. (Ideally something truly stipped down, like https://github.com/dankegel/winezeug/tree/master/cmake_examples/ex7 ) This rings a bell for me. We encountered this link failure once before. A little back history: Our project has two main source folders, common/ and encoder/, each with its own CMakeLists file. Originally we built each folder into a static library and then the root level CMakeLists file linked them into a shared library, static library, or executable. This broke when we introduced yasm-compiled assembly objects into the common/ folder. We begain getting these same exact PIC errors when trying to link the static common.lib and encoder.lib into libx265.so. And this is when we were forced to use cmake's object libraries. Now common/ and encoder/ generate object libraries and the root level CMakeLists links those objects into the final targets and this seemed to work ok. So the yasm objects are somehow the root cause. -- Steve Borho -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
[CMake] linking static libs into shared libs on Linux
Hello, I am the technical lead for x265 (HEVC encoder) and we have used cmake successfully since the beginning of the project. There are a few rough edges that we've had to work around (uninstall rules, yasm support in MSVC, etc) but in general it has worked pretty well. x265 can support a number of bit-depth options, but these are compile time decisions. You must chose at build time whether the encoder will use 8bit, 10bit, or 12bit color samples internally. Recently we've resolved the naming conflicts and thus allow these multiple builds of libx265 to be linked together into a single binary. The way the multilib build process currently works is it creates 8bit/ 10bit/ and 12bit/ folders, then runs cmake in each seperately (with different options) so there are three distinct build folders. It builds 10bit and 12bit first, outputting only static libraries, then finally it builds the 8bit folder and adds the 10bit and 12bit static libraries to the 8bit shared library link target. This works well on Windows and Mac but on x86_64 Linux I get this error: /bin/ld: common/CMakeFiles/common.dir/x86/pixel-a.asm.o: relocation R_X86_64_PC32 against symbol `x265_pw_1' can not be used when making a shared object; recompile with -fPIC (in a nutshell, the 10bit and 12bit static libs are no longer position independent) I figure there are multiple potential solutions for this and would welcome pointers for any of them 1 - is there a way for the project to re-include itself? If so we could have the 8bit build simply build the 10 and 12bit object libraries and link with those directly and bypass the creation of static libraries. This would perhaps be optimal since you would end up with a single Makefile driving the whole build. 2 - is there some invocation that will force cmake to build the static libraries such that they are still safe for linking into a shared library? I've already tried these unsuccessfully: set(CMAKE_POSITION_INDEPENDENT_CODE TRUE) set_target_properties(x265-static PROPERTIES POSITION_INDEPENDENT_CODE TRUE) set_target_properties(x265-static PROPERTIES LINK_FLAGS -fPIC) 3 - is there another intermediate format that cmake can output that preserves -fPIC, or some way to access the object libraries of the other external builds? 4 - is there any post-processing that can be done on the static libs to make them safe for shared libs again (guessing this is not likely) (note: I'm building on Centos 7, but this does not appear to be distribution dependent, and I've tried cmake 2.8.11 and 3.2.3) Thanks -- Steve Borho PS: Our source code is at https://bitbucket.org/multicoreware/x265 -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
On Tuesday, July 07, 2015 14:36:25 Steve Borho wrote: Hello, I am the technical lead for x265 (HEVC encoder) and we have used cmake successfully since the beginning of the project. There are a few rough edges that we've had to work around (uninstall rules, yasm support in MSVC, etc) but in general it has worked pretty well. x265 can support a number of bit-depth options, but these are compile time decisions. You must chose at build time whether the encoder will use 8bit, 10bit, or 12bit color samples internally. Recently we've resolved the naming conflicts and thus allow these multiple builds of libx265 to be linked together into a single binary. The way the multilib build process currently works is it creates 8bit/ 10bit/ and 12bit/ folders, then runs cmake in each seperately (with different options) so there are three distinct build folders. It builds 10bit and 12bit first, outputting only static libraries, then finally it builds the 8bit folder and adds the 10bit and 12bit static libraries to the 8bit shared library link target. This works well on Windows and Mac but on x86_64 Linux I get this error: /bin/ld: common/CMakeFiles/common.dir/x86/pixel-a.asm.o: relocation R_X86_64_PC32 against symbol `x265_pw_1' can not be used when making a shared object; recompile with -fPIC (in a nutshell, the 10bit and 12bit static libs are no longer position independent) I figure there are multiple potential solutions for this and would welcome pointers for any of them 1 - is there a way for the project to re-include itself? If so we could have the 8bit build simply build the 10 and 12bit object libraries and link with those directly and bypass the creation of static libraries. This would perhaps be optimal since you would end up with a single Makefile driving the whole build. 2 - is there some invocation that will force cmake to build the static libraries such that they are still safe for linking into a shared library? I've already tried these unsuccessfully: set(CMAKE_POSITION_INDEPENDENT_CODE TRUE) set_target_properties(x265-static PROPERTIES POSITION_INDEPENDENT_CODE TRUE) set_target_properties(x265-static PROPERTIES LINK_FLAGS -fPIC) 3 - is there another intermediate format that cmake can output that preserves -fPIC, or some way to access the object libraries of the other external builds? 4 - is there any post-processing that can be done on the static libs to make them safe for shared libs again (guessing this is not likely) did you already have a look at the object libraries (created using add_library(... OBJECT...) ? Those basically just build the object files, and can later be used to link into other targets. Alex -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
On 07/07, Alexander Neundorf wrote: On Tuesday, July 07, 2015 14:36:25 Steve Borho wrote: Hello, I am the technical lead for x265 (HEVC encoder) and we have used cmake successfully since the beginning of the project. There are a few rough edges that we've had to work around (uninstall rules, yasm support in MSVC, etc) but in general it has worked pretty well. x265 can support a number of bit-depth options, but these are compile time decisions. You must chose at build time whether the encoder will use 8bit, 10bit, or 12bit color samples internally. Recently we've resolved the naming conflicts and thus allow these multiple builds of libx265 to be linked together into a single binary. The way the multilib build process currently works is it creates 8bit/ 10bit/ and 12bit/ folders, then runs cmake in each seperately (with different options) so there are three distinct build folders. It builds 10bit and 12bit first, outputting only static libraries, then finally it builds the 8bit folder and adds the 10bit and 12bit static libraries to the 8bit shared library link target. This works well on Windows and Mac but on x86_64 Linux I get this error: /bin/ld: common/CMakeFiles/common.dir/x86/pixel-a.asm.o: relocation R_X86_64_PC32 against symbol `x265_pw_1' can not be used when making a shared object; recompile with -fPIC (in a nutshell, the 10bit and 12bit static libs are no longer position independent) I figure there are multiple potential solutions for this and would welcome pointers for any of them 1 - is there a way for the project to re-include itself? If so we could have the 8bit build simply build the 10 and 12bit object libraries and link with those directly and bypass the creation of static libraries. This would perhaps be optimal since you would end up with a single Makefile driving the whole build. 2 - is there some invocation that will force cmake to build the static libraries such that they are still safe for linking into a shared library? I've already tried these unsuccessfully: set(CMAKE_POSITION_INDEPENDENT_CODE TRUE) set_target_properties(x265-static PROPERTIES POSITION_INDEPENDENT_CODE TRUE) set_target_properties(x265-static PROPERTIES LINK_FLAGS -fPIC) 3 - is there another intermediate format that cmake can output that preserves -fPIC, or some way to access the object libraries of the other external builds? 4 - is there any post-processing that can be done on the static libs to make them safe for shared libs again (guessing this is not likely) did you already have a look at the object libraries (created using add_library(... OBJECT...) ? Those basically just build the object files, and can later be used to link into other targets. We use object libraries internally, but I don't know how the 8bit build can access the object libraries from the 10bit or 12bit builds. -- Steve Borho -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
On Tue, Jul 7, 2015 at 2:25 PM, Steve Borho st...@borho.org wrote: We're already adding -fPIC to the compile flags for the two object libraries. This way one set of objects can be used to output the shared library and the static library. So the C++ files are already compiled with -fPIC. It is only a problem when we need to link a static library from one build into the shared library of another build. The static library itself no longer supports PIC. Really? I didn't think ar was in the habit of modifying machine code like that. At this point maybe the thing to do is make a toy example that reproduces the problem. (Ideally something truly stipped down, like https://github.com/dankegel/winezeug/tree/master/cmake_examples/ex7 ) - Dan -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
On Tue, Jul 7, 2015 at 12:36 PM, Steve Borho st...@borho.org wrote:
/bin/ld: common/CMakeFiles/common.dir/x86/pixel-a.asm.o: relocation
R_X86_64_PC32 against symbol `x265_pw_1' can not be used when making a
shared object; recompile with -fPIC
...
2 - is there some invocation that will force cmake to build the static
libraries such that they are still safe for linking into a shared
library?
I've already tried these unsuccessfully:
set(CMAKE_POSITION_INDEPENDENT_CODE TRUE)
set_target_properties(x265-static PROPERTIES POSITION_INDEPENDENT_CODE
TRUE)
set_target_properties(x265-static PROPERTIES LINK_FLAGS -fPIC)
3 - is there another intermediate format that cmake can output that
preserves -fPIC, or some way to access the object libraries of the
other external builds?
Try
set_target_properties(x265-static PROPERTIES COMPILE_FLAGS -fPIC)
Alternately, something like
set (CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS} -fPIC)
when compiling the files in question.
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
Re: [CMake] linking static libs into shared libs on Linux
On 07/07, Dan Kegel wrote:
On Tue, Jul 7, 2015 at 12:36 PM, Steve Borho st...@borho.org wrote:
/bin/ld: common/CMakeFiles/common.dir/x86/pixel-a.asm.o: relocation
R_X86_64_PC32 against symbol `x265_pw_1' can not be used when making a
shared object; recompile with -fPIC
...
2 - is there some invocation that will force cmake to build the static
libraries such that they are still safe for linking into a shared
library?
I've already tried these unsuccessfully:
set(CMAKE_POSITION_INDEPENDENT_CODE TRUE)
set_target_properties(x265-static PROPERTIES POSITION_INDEPENDENT_CODE
TRUE)
set_target_properties(x265-static PROPERTIES LINK_FLAGS -fPIC)
3 - is there another intermediate format that cmake can output that
preserves -fPIC, or some way to access the object libraries of the
other external builds?
Try
set_target_properties(x265-static PROPERTIES COMPILE_FLAGS -fPIC)
Alternately, something like
set (CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS} -fPIC)
when compiling the files in question.
sadly that didn't help.
We're already adding -fPIC to the compile flags for the two object
libraries. This way one set of objects can be used to output the shared
library and the static library. So the C++ files are already compiled
with -fPIC. It is only a problem when we need to link a static library
from one build into the shared library of another build. The static
library itself no longer supports PIC.
--
Steve Borho
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake
