Re: [CMake] Redundant linking when modifying shared libraries
Sounds good. The FAQ has a question "Is there a way to skip checking of dependent libraries when compiling?" - perhaps you can add one just before that, something like "Is there a way to reduce the amount of linking when building a large project?".or similar. Thanks again, itay On Thu, Feb 14, 2019 at 11:26 PM Robert Maynard wrote: > > I agree that we should document this property better. > I recommend looking at the CMake wiki ( > https://gitlab.kitware.com/cmake/community/wikis/home ) and thinking > maybe adding a new recipe for `optimizing redundant linking`. > > On Thu, Feb 14, 2019 at 3:11 PM Itay Chamiel wrote: > > > > On Thu, Feb 14, 2019 at 8:08 PM Robert Maynard > > wrote: > > > By default CMake wants to get a correct build 100% of the time. There > > > is nothing to stop people from having functions defined in a .cxx file > > > with no corresponding header, and using manual forward deceleration is > > > used in a consuming library/executable. By setting > > > `LINK_DEPENDS_NO_SHARED` to `True` you will convert what was a link > > > time error ( renaming the function in the .cxx ) to a run time error. > > > > First of all, thanks for the answer - it's good to understand the > > rationale and to know that in our use case we can safely use this > > setting. > > > > On the question of whether to set this as the default, I understand > > your position and I personally disagree - but do not wish to start a > > debate. However I will say that this simple change has, in a common > > case, reduced my build time from 5 minutes to 15 seconds. At the very > > least could you consider steps to make this setting more known and > > accessible? (No one on Stack Overflow knew about it, and I think > > that's saying something.) Perhaps put it in a FAQ, and/or more > > prominently in the documentation. > > > > Kind regards, > > > > itay > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: > > http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more > > information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at > > http://www.kitware.com/opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > https://cmake.org/mailman/listinfo/cmake -- Itay Chamiel, Software Engineer itay.cham...@orcam.com | Office: +972-2-591-7800, ext. 7814 OrCam Technologies www.orcam.com -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Redundant linking when modifying shared libraries
I agree that we should document this property better. I recommend looking at the CMake wiki ( https://gitlab.kitware.com/cmake/community/wikis/home ) and thinking maybe adding a new recipe for `optimizing redundant linking`. On Thu, Feb 14, 2019 at 3:11 PM Itay Chamiel wrote: > > On Thu, Feb 14, 2019 at 8:08 PM Robert Maynard > wrote: > > By default CMake wants to get a correct build 100% of the time. There > > is nothing to stop people from having functions defined in a .cxx file > > with no corresponding header, and using manual forward deceleration is > > used in a consuming library/executable. By setting > > `LINK_DEPENDS_NO_SHARED` to `True` you will convert what was a link > > time error ( renaming the function in the .cxx ) to a run time error. > > First of all, thanks for the answer - it's good to understand the > rationale and to know that in our use case we can safely use this > setting. > > On the question of whether to set this as the default, I understand > your position and I personally disagree - but do not wish to start a > debate. However I will say that this simple change has, in a common > case, reduced my build time from 5 minutes to 15 seconds. At the very > least could you consider steps to make this setting more known and > accessible? (No one on Stack Overflow knew about it, and I think > that's saying something.) Perhaps put it in a FAQ, and/or more > prominently in the documentation. > > Kind regards, > > itay > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Redundant linking when modifying shared libraries
On Thu, Feb 14, 2019 at 8:08 PM Robert Maynard wrote: > By default CMake wants to get a correct build 100% of the time. There > is nothing to stop people from having functions defined in a .cxx file > with no corresponding header, and using manual forward deceleration is > used in a consuming library/executable. By setting > `LINK_DEPENDS_NO_SHARED` to `True` you will convert what was a link > time error ( renaming the function in the .cxx ) to a run time error. First of all, thanks for the answer - it's good to understand the rationale and to know that in our use case we can safely use this setting. On the question of whether to set this as the default, I understand your position and I personally disagree - but do not wish to start a debate. However I will say that this simple change has, in a common case, reduced my build time from 5 minutes to 15 seconds. At the very least could you consider steps to make this setting more known and accessible? (No one on Stack Overflow knew about it, and I think that's saying something.) Perhaps put it in a FAQ, and/or more prominently in the documentation. Kind regards, itay -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Redundant linking when modifying shared libraries
> I wonder why this isn't the default behavior By default CMake wants to get a correct build 100% of the time. There is nothing to stop people from having functions defined in a .cxx file with no corresponding header, and using manual forward deceleration is used in a consuming library/executable. By setting `LINK_DEPENDS_NO_SHARED` to `True` you will convert what was a link time error ( renaming the function in the .cxx ) to a run time error. On Thu, Feb 14, 2019 at 12:11 PM Itay Chamiel wrote: > > On Thu, Feb 14, 2019 at 12:39 PM Craig Scott wrote: > > I think you might be looking for the LINK_DEPENDS_NO_SHARED target property > > (or more likely its associated CMAKE_LINK_DEPENDS_NO_SHARED variable). > > After my previous response I experimented a little more, and I got it > to work. My mistake was that I needed to add the configuration line to > the executable target, not the lib. So in my example, add the > following line to the Demo/CMakeLists.txt file: > > set_target_properties(helloDemo PROPERTIES LINK_DEPENDS_NO_SHARED true) > > ..and it works like a charm. I wonder why this isn't the default > behavior; can you think of any scenario where this would produce > invalid results? > > Thanks for your help! > > itay > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Redundant linking when modifying shared libraries
On Thu, Feb 14, 2019 at 12:39 PM Craig Scott wrote: > I think you might be looking for the LINK_DEPENDS_NO_SHARED target property > (or more likely its associated CMAKE_LINK_DEPENDS_NO_SHARED variable). After my previous response I experimented a little more, and I got it to work. My mistake was that I needed to add the configuration line to the executable target, not the lib. So in my example, add the following line to the Demo/CMakeLists.txt file: set_target_properties(helloDemo PROPERTIES LINK_DEPENDS_NO_SHARED true) ..and it works like a charm. I wonder why this isn't the default behavior; can you think of any scenario where this would produce invalid results? Thanks for your help! itay -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Redundant linking when modifying shared libraries
On 2/14/19 12:38 PM, Craig Scott wrote: I think you might be looking for the LINK_DEPENDS_NO_SHARED target property (or more likely its associated CMAKE_LINK_DEPENDS_NO_SHARED variable). Following with my example, I've added the following line to the file Hello/CMakeLists.txt: set_target_properties(Hello PROPERTIES LINK_DEPENDS_NO_SHARED true) However it has no effect. Touching the file Hello/hello.cpp will still cause the "Linking CXX executable helloDemo" step to be performed. I've also tried adding the same line to the CMakeLists.txt file in the root directory, it also had no effect. Thank you, itay -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Redundant linking when modifying shared libraries
On Thu, Feb 14, 2019 at 9:24 PM Itay Chamiel wrote:
> Hi,
>
> I've asked this question on Stack Overflow almost a year ago with no
> useful responses (with the same topic if you wish to search for it), so I'm
> trying my luck here.
>
> I work on a large commercial C++ project comprised of a couple dozen
> dynamically linked shared libraries, each of which has many associated unit
> tests. Many libs are also dependent on other libs because a lib for some
> specific functionality will use the code from one of the more common libs.
> And finally of course there are the production executables which depend on
> the libs.
>
> There is no question that a change in the API (a header file) of some core
> common lib should trigger a major recompilation of nearly the entire
> system. But typically there is only a change in some function's
> implementation, and the only file compiled is the modified .cpp file, and
> in theory just the modified lib would need to be linked - thanks to dynamic
> linking there should be no need to relink anything else. But CMake goes
> ahead and does it anyway: after relinking the lib it relinks all the unit
> tests associated with that lib. Then it relinks all the libs in that lib's
> dependency tree and all their unit tests. Finally it relinks the production
> executables. Due to the scale of the project this takes a lot of precious
> time.
>
> I have reproduced this behavior with a simple project based on the example
> at cmake.org/examples (comments removed for brevity and lib type changed
> to shared). My system is Ubuntu 16 on an Intel PC and my CMake version is
> 3.5.1.
>
> Start with an empty directory. Create subdirectories Demo and Hello and
> then create these files:
>
> * CMakeLists.txt
>
> cmake_minimum_required (VERSION 2.8.11)
> project (HELLO)
> add_subdirectory (Hello)
> add_subdirectory (Demo)
>
> * Demo/CMakeLists.txt
>
> add_executable (helloDemo demo.cpp)
> target_link_libraries (helloDemo LINK_PUBLIC Hello)
>
> * Demo/demo.cpp
>
> #include "hello.h"
> int main() { hello(); }
>
> * Hello/CMakeLists.txt
>
> add_library (Hello SHARED hello.cpp)
> target_include_directories (Hello PUBLIC ${CMAKE_CURRENT_SOURCE_DIR})
>
> * Hello/hello.h
>
> void hello();
>
> * Hello/hello.cpp
>
> #include
> void hello() { printf("hello!\n"); }
>
> * now run the commands:
>
> mkdir build
> cd build
> cmake ../
> make
>
> * You may now execute Demo/helloDemo and see hello!.
>
> Now, touch Hello/hello.cpp and make again. You will see that libHello is
> built and linked as expected, but then the helloDemo executable is also
> relinked ("Linking CXX executable helloDemo"). The latter step is
> completely redundant; even if hello.cpp is modified to print a different
> string the relinked executable remains binary identical.
>
> Is there a way to prevent these redundant build actions? This would be a
> huge time saver for us.
>
I think you might be looking for the LINK_DEPENDS_NO_SHARED
target
property (or more likely its associated CMAKE_LINK_DEPENDS_NO_SHARED
variable).
--
Craig Scott
Melbourne, Australia
https://crascit.com
Get the hand-book for every CMake user: Professional CMake: A Practical
Guide
