Thanks for the suggestions Clinton. The idea of using a custom target to invoke CMake's --build feature on a sub-project was the piece I needed. When combined with ExternalProject, this gives me the behavior I was after. I use ExternalProject to define a separate build for each compiler and some additional custom targets ensure a build from the top level picks up changes to files in the sub projects. It's still a little bit awkward, but definitely workable.
The packaging part was best handled by defining each package in its own ExternalProject too and making use of CPACK_INSTALL_CMAKE_PROJECTS to define the components to pull in from the various builds. This meant I could keep the structure of everything relatively close to being as though it was all part of a single unified build and not have to bother with importing/exporting targets, etc. On Mon, Aug 26, 2013 at 11:19 PM, <[email protected]> wrote: > > > ------------------------------ > > Hi. First, apologies for the length. If you are not interested in mixing > different compilers and generating multiple packages all within one build, > you can probably stop reading now. > > After trying various approaches and not being entirely satisfied with any, > I thought I'd seek the collective wisdom on this list. The details below > define the scenario, the main problems faced and some things that have been > tried to address them. Comments, ideas or suggestions would be most welcome. > > SCENARIO: > > We have a large code base for which different components need to be built > with different compilers. Some parts are desktop software (applications, > libraries) and others are firmware for embedded devices. All are pulled > together into a small number of packages using CPack. Some of those > packages share common items and there is no correlation between compilers > and packages. Most items are included in more than one of the packages. For > instance, firmware might be included in all packages, a particular desktop > application or library in a subset of the packages, etc. On a given > platform, some packages will have one format and other packages will have a > different format (eg a mix of RPM and tarball). We create packages for at > least Windows and Linux. > > Most of the source code is in a single git repository and after testing > different strategies, this is an arrangement we want to keep. There are a > couple of components being built using CMake's ExternalProject feature, > either because they use a different build system that is currently too > complex for us to change to CMake or the source code comes from a 3rd party. > > > PROBLEMS: > > 1. CMake assumes you only want to use one compiler for an entire build > tree. > 2. CPack assumes you are only building one package. Some package > generators allow you to do a component-based setup where multiple packages > can be generated, but this is not supported for all package generators (eg > the NSIS package generator won't produce multiple packages). > > > APPROACHES TRIED (feedback welcome): > > The first of the above two issues is the more problematic. I've spent > considerable time experimenting with ExternalProject as a potential way of > addressing various issues, including the "one compiler per build tree" > restriction. What I found was that dependency handling and integration with > CPack becomes much more complicated, and in some cases has deficiencies not > present in a single unified CMake build. For example, once you have > successfully built an external project, the top level build won't rebuild > it again unless something about its configuration changes. Thus, if you > subsequently make changes to a source file, simply building the top level > project won't work because it still thinks the sub project is up to date. > This is because ExternalProject uses time stamps rather than forwarding the > build request to the sub project to let it decide if anything needs to be > built. This kinda makes sense, since an ExternalProject is meant to > encompass everything from download through to install, but this is not how > we are trying (and needing) to use it. ExternalProject starts to become > less useful when applied to source code you are actively working on. > > Effectively, what we really need is the ability to let add_subdirectory() > create a whole new build with a potentially different compiler, but still > have it aware of all the other targets and dependencies and also to > propagate all its own the targets, dependencies, etc. up to the parent. We > experimented with exporting targets from the ExternalProject's using the > export features of the install command, but this was messy and ultimately > still suffers from the "isn't aware of changes to the source" problem. It > also has implications for stripping of targets (for an example, see > http://public.kitware.com/Bug/view.php?id=14157 ). Packaging also gets > much more cumbersome when ExternalPackage is involved. > > For the second problem, my current workaround is to create a custom target > which re-configures and re-builds the project for each package I need to > create. The targets only need to get built once, but the package details > change each time and a different package gets built. In essence, the build > calls itself with a set of different configurations, which seemed a bit > mind-bending at first, but was fairly simple to implement and didn't seem > to have much in the way of negative side effects. The approach works > reliably, but it feels like a roundabout solution to something that > CMake/CPack should be able to handle much more cleanly. In a similar vein > to the suggestion above, it would be really nice if add_subdirectory() had > the ability to define a package that was not affected or did not affect > other directories. Thus, you could have one package per sub directory. Any > common variables could be defined at the parent level and would propagate > down to the sub directories. Alas, this is not something that > add_subdirectory() currently supports. > > I'd be interested in hearing what other people have done or would do to > address the above problems for the specified scenario. I'm happy to discuss > in more detail things I've tried if people are interested, but hopefully > the above will stimulate some ideas or discussion about how these two > problems can be better handled with CMake/CPack. These are great tools for > which I'm sure these limitations can be overcome, either through methods I > haven't considered or improvements to the tools themselves. > > > This idea may or may not work well for you. > > To replace your ExternalProject, with a method that has deep build > dependencies, you can try this: > > Do an execute process so the configure step of the sub project happens at > the same time as the configure of the parent project: > > set(sub_BINARY_DIR "${CMAKE_CURRENT_BINARY_DIR}/sub" CACHE INTERNAL "") > set(sub_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/sub" CACHE INTERNAL "") > > execute_process(WORKING_DIRECTORY ${sub_BINARY_DIR} > COMMAND ${CMAKE_COMMAND} -G "Unix Makefiles" > # standard cmake options > -D CMAKE_C_COMPILER:FILEPATH=${CMAKE_C_COMPILER} > -D CMAKE_OSX_ARCHITECTURES:STRING=${CMAKE_OSX_ARCHITECTURES} > -D CMAKE_C_FLAGS:STRING=${CMAKE_C_FLAGS} > -D CMAKE_BUILD_TYPE:STRING=${CMAKE_BUILD_TYPE} > # 3rd party options > ..... > # the source tree > "${sub_SOURCE_DIR}" > ) > > Then the build step can be handled like this ( you may need to fill out > dependencies or configuration type ): > > add_custom_target(sub ALL > COMMAND "${CMAKE_COMMAND}" > --build "${sub_BINARY_DIR}" > COMMENT "Building sub..." > VERBATIM > ) > > > > For the packaging part, you can create multiple cpack config files so you > do not have to regenerate a single config file. > To use a specific config file, use the --config option to cpack. To > select parts of the software to go into certain packages, you can use > components, and each cpack config file can list which components to include > in that package. > > > To include the disconnected sub projects in packaging, I am guessing you > can try something like this: > install(CODE " > include(\"${CMAKE_CURRENT_BINARY_DIR}/sub/cmake_install.cmake\") > ") > > Clint > > -- Craig Scott Melbourne, Australia
-- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake
