Re: [CMake] CMake Project Generation Speedup
The best advice I had for myself: profile the code and tweak it. The last I looked, and that was a while ago, the generators could use a threaded write queue so that the file output would overlap with file generation. Those probably could be memory mapped files, too, to further minimize the overhead: writing to mapped pages that are dumped to disk via DMA (that’s how page flushes work) is cheaper than copying all the data in little chunks into a small buffer in the file output code, and then once again into the page that ends up flushed to disk. Never mind that the write latencies are on the hot path, where they don’t belong. If generators could be const-correct (not writing anywhere into the Makefile data structures), they could all be parallelized as well – but I’m not sure whether that’s a minor hack or a major rework. Cheers, Kuba > 20 mars 2019 kl. 16:53 skrev J. Caleb Wherry : > > Did anything ever come of this? > > I am in a similar boat: we have >800 targets on our full build (native C++, > Managed C++, C#, Java, CUDA, etc) and the majority of the time for the > configure/generate steps takes place in the generate step (>70%). > > I understand there is a lot of IO since the generate step has to write the > project files and filters for each C++ project (the majority of our projects) > for VS generators (what we use). I'm just looking to see if there is anything > to look at or potentially speedup up the generate step besides "get a faster > drive". > > Thanks! > -Caleb > >> On Thu, Nov 17, 2016 at 11:15 AM Damian wrote: >> Hi all, >> >> We are still in the process of switching our large Make-based build to >> CMake. One of the issues we're running into is the time it takes to reparse >> and regenerate the CMake project (whether ninja, VS, or make) after >> touching any CMake file. To give you an idea, we have about 1000 targets and >> that takes a good 2 min for CMake to rerun. >> >> Are there any plans to speed this up? Maybe parallelize it in some way or do >> a better job regenerating only what needs regenerating? Is there anything we >> can do on our side to reduce our regeneration times? >> >> For example, if using a VS generator, each directory in the source that has >> a CMakeLists.txt gets a .vcproj and .sln generated. Ideally, if I touch one >> of those CMakeLists.txt, only that .sln/.vcproj would get regenerated. >> >> Thanks for any help. >> -- >> >> Powered by www.kitware.com >> >> Please keep messages on-topic and check the CMake FAQ at: >> http://www.cmake.org/Wiki/CMake_FAQ >> >> Kitware offers various services to support the CMake community. For more >> information on each offering, please visit: >> >> CMake Support: http://cmake.org/cmake/help/support.html >> CMake Consulting: http://cmake.org/cmake/help/consulting.html >> CMake Training Courses: http://cmake.org/cmake/help/training.html >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Follow this link to subscribe/unsubscribe: >> http://public.kitware.com/mailman/listinfo/cmake > > > -- > J. Caleb Wherry > Scientific Software Engineer > > http://www.calebwherry.com > +1 (615) 708-5651 > calebwhe...@gmail.com > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
[CMake] Is there an idiomatic way of setting platform specific compiler options in a generic way?
I am developing platform support for an embedded target (Z8 Encore!), and it mostly works – but I’d like to make it easier to set the compiler and linker flags, as there’s a ton of them: and they are nothing like GNU flags, so they only apply when building for that particular target. Thus a question: is there a preferred way for the user to set platform-specific flags without having to know them exactly? I’m thinking of having some sort of platform-agnostic means of selecting compile and link flags, and then translating those for the platform the target is generated for. Specifically, I want to avoid having the following in CMakeLists for the project: if("${COMPILER_ID}" strequal "ez8cc") target_compile_options(target1 PRIVATE -const:RAM) elseif("${COMPILER_ID}" strequal "gcc" and "${CMAKE_SYSTEM_PROCESSOR}" strequal "somecpu") target_compile_definitions(target1 PRIVATE ram rom far near) # those shall be empty endif() I’d like to do something like this instead: set_target_properties(target1 PROPERTIES OPT_CONST RAM OPT_PACK TIGHT …) Then, a process_target_opts() function called at the end of CMakeLists would iterate the targets, and for each target call process_${COMPILER_ID}_opts() (if such function would be present or issue a warning otherwise). That function then converts the OPT_ options to platform-specific flags, based on cpu and/or compiler. Is it something that would be more-or-less idiomatic, or is there another preferred way of doing it? I really don’t want to list the options manually for each target/platform combo, as they’d differ quite a bit, and I have lots of targets (hundreds). I’d appreciate any hints. Cheers, Kuba Ober -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
Re: [CMake] Remove folders created by install
Use something to read the manifest, extract paths from it, remove the duplicates, then iterate them, and remove the ones that are empty. It’d be a bad idea probably to remove ones that are not empty. Cheers, Kuba > 16 feb. 2019 kl. 09:47 skrev Felix Crazzolara : > > Hi everyone > > For my smaller projects I'd like to have 'uninstall' functionality. To remove > installed files I can call: > > xargs rm < build/install_manifest.txt > > Unfortunately this won't delete any folders generated by the installation. Is > there a different file that keeps track of the created directories, or what > is the recommended way to implement such functionality? > > Example: > Suppose that I install _some_header.hpp in > /include// using the command install(TARGETS > EXPORT -targets ARCHIVE DESTINATION lib > PUBLIC_HEADER DESTINATION include/) then I want not only to > remove /include//_some_header.hpp, but > also the directory /include//. > > Cheers, > > Felix Crazzolara > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > https://cmake.org/mailman/listinfo/cmake -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake
[CMake] Generator expressions and rule variables
Hi, I'm trying to integrate cmake with a set of build tools whose command lines look nothing like standard gnu tools. Unfortunately, both the generator expressions and the rule variables seem very limited: I have to do some custom massaging of the arguments that get passed to the compiler/linker. I've resorted to using cmake in script mode as a proxy to the compiler and linker, passing all the rule variables verbatim, and then having the script call the compiler or linker. This suffers from problems with escaping of spaces, at the very least. It seems like those compilation commands would need nested lists (a hypothetical construct thus far): the outer list would be the list of commands - as it is now, the inner list would be the list of arguments, and they'd be properly escaped etc. Some of it could be solved if COMMAND_EXPAND_LISTS could be somehow applied to normal targets, and thus to the underlying CMAKE__COMPILE_OBJECT, etc. Simplifying, thus far I set CMAKE_C_COMPILE_OBJECT to ${CMAKE_COMMAND} -DV1= ... -DVn= -P ${CMAKE_CURRENT_LIST_DIR}/helper.cmake. The helper does the rest. It's brittle because of poor escaping support. I could also use C_COMPILER_LAUNCHER with same effect but it couldn't be a cmake script, since cmake doesn't understand "cmake -D... -P script.cmake -- arbitrary arguments" (it could without any trouble - it's a small patch; would it be something worthwhile to add?). The problem is that I don't really need the overhead of invoking cmake or another script/process for every compiled/linked output. For both make and ninja generators, all I'd need is to generate a custom target command that bypasses the result of the built-in generator. But I also want my CMakeLists to look familiar and be compatible with additional generators, thus having a custom "add_my_executable" function wouldn't work, and overriding "add_executable" to modify some targets, and pass others directly to "add_executable_" (the undocumented handle to the overridden function) seems like a big hack. So, what I'd want to do is to have a function that can be invoked for every target, before the target is passed to the generator, and that could inspect the target and modify its properties and override variables in its scope. In my case, I'd write this function so that each target would get its own "CMAKE_C_COMPILE_OBJECT" or "CMAKE_C_LINK_EXECUTABLE", fully expanded, without any generator expressions nor rule variables. Does cmake already offer any hook like it? A cursory examination of the source code doesn't seem to indicate so (btw: the code is clear and easy to follow, so I don't have high hopes...). And if not, would it be a worthwhile addition? It'd obviate the need for both generator expressions and rule variables - personally I think that they are half-baked hacks, with rule variables being even more limited and really only meshing with gnu-like tools. I imagine that to speed things up, a pre-filter could be applied - say that "my_function" should be invoked for all executable targets, for C language and foocc compiler: "CMAKE_ADD_FILTER(add_executable, my_function, LANGUAGE=c COMPILER_ID=foocc)". For every invocation of `add_executable`, CMAKE would check if the filters match, and if so it'd invoke the function, which then could modify target properties and variables visible to the generator (i.e. it could regenerate CMAKE_C_COMPILE_OBJECT). Another approach could be to have said function act like COMPILER_LAUNCHER, and we'd then have something like: "CMAKE_ADD_COMPILER_LAUNCHER(my_function, LANGUAGE=c, COMPILER_ID=foocc, TYPE=executable)" Cheers, Kuba Ober -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: https://cmake.org/mailman/listinfo/cmake