Re: [cmake-developers] CMake daemon-mode
Hi Brad, Am 17.06.2016 22:09 schrieb "Brad King" : > Then I applied the patches with some revisions: > > cmGeneratorTarget: Adopt Fortran module directory generation > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=49f10f0d > > cmLocalGenerator: Add method to get Fortran-specific compiler flags > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=5467e794 > > cmLocalGenerator: Adopt target compile flag generation > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3c488ce8 > > cmExtraSublimeTextGenerator: Use GetTargetCompileFlags > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=802b36fb > > These have been merged to `next` for testing. Wonderful! Thanks for improving the patches I'll rebase my branch on top of these as soon as they hit master. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/10/2016 12:31 PM, Tobias Hunger wrote: > I did try to do the changes you requested. The branch is here: > https://github.com/hunger/CMake/commits/compileflags Thanks. I did some refactoring to help address the path conversion problem: Refactor Makefile/Ninja tool working directory storage https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=0392f72b Then I applied the patches with some revisions: cmGeneratorTarget: Adopt Fortran module directory generation https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=49f10f0d cmLocalGenerator: Add method to get Fortran-specific compiler flags https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=5467e794 cmLocalGenerator: Adopt target compile flag generation https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=3c488ce8 cmExtraSublimeTextGenerator: Use GetTargetCompileFlags https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=802b36fb These have been merged to `next` for testing. > Next patch moves the AddFortranFlags code. This will need some careful > review, as it removes a path conversion that were present in the > original code (in two places). I could just not figure out how > to convert in the new place. I replaced that commit with a different one that preserves the path conversion. The implementation needs to be in cmLocalCommonGenerator in order to have access to the needed information to do the conversion. I used virtual dispatch to achieve this. On other generators we do not need these flags anyway. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Brad, I just rebased this branch on top of current master and pushed it. There were some conflicts. Best Regards, Tobias On Mon, Jun 13, 2016 at 4:16 PM, Brad King wrote: > On 06/10/2016 12:31 PM, Tobias Hunger wrote: >> I did try to do the changes you requested. The branch is here: >> >> https://github.com/hunger/CMake/commits/compileflags > > Thanks. I'm about to go on travel but should hopefully be able to > look at that again when I return. > > -Brad > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/10/2016 12:31 PM, Tobias Hunger wrote: > I did try to do the changes you requested. The branch is here: > > https://github.com/hunger/CMake/commits/compileflags Thanks. I'm about to go on travel but should hopefully be able to look at that again when I return. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Brad, On Thu, Jun 9, 2016 at 7:57 PM, Brad King wrote: > On 06/09/2016 09:27 AM, Brad King wrote: >>> https://github.com/hunger/CMake/commit/bc060a44b6b2c2281ffe99815163ba02ec835dd6 >> Good. I need to review that one more deeply but should be able to integrate >> it. > > I split out some incremental commits toward that: > > cmCommonTargetGenerator: De-duplicate CMAKE_BUILD_TYPE lookup > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=de4ee088 > > cmLocalGenerator: Adopt GetFrameworkFlags method > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=70d3bf85 > > cmLocalGenerator: Move GetFrameworkFlags implementation to private helper > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d9613b96 > > Please rebase on that (at least once it is in master). > > For the main change, please also move over AddFortranFlags, perhaps > becoming cmLocalGenerator::GetFortranFlags. The result should be > that cmCommonTargetGenerator::GetFlags moves into the new > cmLocalGenerator::GetTargetCompileFlags method except for the caching. > That will preserve flag order for GetFlags. > > For the Xcode part of the changes, we cannot use the full > GetTargetCompileFlags. The reason is that for Xcode some of the > flags, like the architecture settings, become .pbxproj file properties > instead of raw flags. It may be simplest to just drop that part of > the change for now. Or, we need to factor out an intermediate helper > that can be shared by GetTargetCompileFlags and Xcode. I did try to do the changes you requested. The branch is here: https://github.com/hunger/CMake/commits/compileflags Some comments on the individual commits (and a direct link): This moves the fortran module directory creation: https://github.com/hunger/CMake/commit/eb417fef8bd1c91041a211899a2a7984aa75774d Next patch moves the AddFortranFlags code. This will need some careful review, as it removes a path conversion that were present in the original code (in two places). I could just not figure out how to convert in the new place. https://github.com/hunger/CMake/commit/2d5f6c673cc8a39f8590acfc23d4a6d5af8900a7 Then I add the cmLocalGenerator::GetTargetCompileFlags method I want. https://github.com/hunger/CMake/commit/9be6beddfe2c2eac6b61d572ee3e5131b0f0b30e Finally I convert the Sublime generator over to the new code: https://github.com/hunger/CMake/commit/deb0793a243c266aee00d4f6e07685a5268f3dae As I said, the second one will need some special care during the review. I did run all unit tests with all patches applied, and all pass on my system. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/09/2016 04:24 PM, Tobias Hunger wrote: >> Please rebase on that (at least once it is in master). > > How long does it usually take to get something into master? Typically a (business) day or two, but sometimes longer depending on trouble with testing in 'next' on our dashboards. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Brad, Am 09.06.2016 19:57 schrieb "Brad King" : > > On 06/09/2016 09:27 AM, Brad King wrote: > >> https://github.com/hunger/CMake/commit/bc060a44b6b2c2281ffe99815163ba02ec835dd6 > > Good. I need to review that one more deeply but should be able to integrate it. > > I split out some incremental commits toward that: > > cmCommonTargetGenerator: De-duplicate CMAKE_BUILD_TYPE lookup > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=de4ee088 > > cmLocalGenerator: Adopt GetFrameworkFlags method > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=70d3bf85 > > cmLocalGenerator: Move GetFrameworkFlags implementation to private helper > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d9613b96 > > Please rebase on that (at least once it is in master). How long does it usually take to get something into master? I do rebase rather often on top of master right anyway. I do want to stay current as I hate running into merge conflicts after weeksof solitary work in my little ivory tower of code:) > For the main change, please also move over AddFortranFlags, perhaps > becoming cmLocalGenerator::GetFortranFlags. The result should be > that cmCommonTargetGenerator::GetFlags moves into the new > cmLocalGenerator::GetTargetCompileFlags method except for the caching. > That will preserve flag order for GetFlags. I will give that a try. > For the Xcode part of the changes, we cannot use the full > GetTargetCompileFlags. The reason is that for Xcode some of the > flags, like the architecture settings, become .pbxproj file properties > instead of raw flags. It may be simplest to just drop that part of > the change for now. Or, we need to factor out an intermediate helper > that can be shared by GetTargetCompileFlags and Xcode. I think I will just drop that part then. I do not have a Mac handy to test anyway. > Thanks, > -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/09/2016 09:27 AM, Brad King wrote: >> https://github.com/hunger/CMake/commit/bc060a44b6b2c2281ffe99815163ba02ec835dd6 > Good. I need to review that one more deeply but should be able to integrate > it. I split out some incremental commits toward that: cmCommonTargetGenerator: De-duplicate CMAKE_BUILD_TYPE lookup https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=de4ee088 cmLocalGenerator: Adopt GetFrameworkFlags method https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=70d3bf85 cmLocalGenerator: Move GetFrameworkFlags implementation to private helper https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=d9613b96 Please rebase on that (at least once it is in master). For the main change, please also move over AddFortranFlags, perhaps becoming cmLocalGenerator::GetFortranFlags. The result should be that cmCommonTargetGenerator::GetFlags moves into the new cmLocalGenerator::GetTargetCompileFlags method except for the caching. That will preserve flag order for GetFlags. For the Xcode part of the changes, we cannot use the full GetTargetCompileFlags. The reason is that for Xcode some of the flags, like the architecture settings, become .pbxproj file properties instead of raw flags. It may be simplest to just drop that part of the change for now. Or, we need to factor out an intermediate helper that can be shared by GetTargetCompileFlags and Xcode. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/08/2016 10:11 AM, Brad King wrote: > I had to revert it from 'next' because it caused several LinkFlags > test failures. [snip] On 06/08/2016 12:02 PM, Tobias Hunger wrote: > https://github.com/hunger/CMake/commit/66acf05bc1737211f88b6ad13781c791f1a7bce4 > > has an update. None of the tests fail for me with that. Thanks. I applied that one and merged to 'next' yesterday for testing. It passed this time: cmLocalGenerator: Pass configuration to GetTargetFlags https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=b0d3e693 > https://github.com/hunger/CMake/commit/bc060a44b6b2c2281ffe99815163ba02ec835dd6 > > Adds a method to retrieve compile flags from cmLocalGenerator. Good. I need to review that one more deeply but should be able to integrate it. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/08/2016 01:24 AM, Tobias Hunger wrote: >> cmLocalGenerator: Pass configuration to GetTargetFlags >> https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=fe0d2241 > > Great, thanks! I had to revert it from 'next' because it caused several LinkFlags test failures. The reason is that it dropped UpperCase() on the config name when called from the Ninja generator. Please revise accordingly in combination with the refactoring discussed in the thread you started dedicated to that method. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/08/2016 04:05 AM, Stephen Kelly wrote: > I have rebased it and force pushed my github branch (there are a few > rebasing mistakes which I'll remove later). Thanks. > We hit issues at work that different clang-format versions have > different behavior. There we have the luxury of providing particular > binaries that every developer uses to get identical results and no diff > confusion. > > I don't know how to solve that in CMake though. It could become an issue > in the future when some contributors use clang-3.9 and others 3.8. Maybe > clang-format binaries should be in the repo. Right now we're using clang-format 3.8. We'll worry about such differences when they become a problem. > Also, it seems that clang-format is not run by CI. The ProductBuild > stuff doesn't seem to be clang formatted. Yeah, we haven't gotten that set up yet. Yesterday I added a topic to 'next' to fix that formatting. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/07/2016 11:42 AM, Daniel Pfeifer wrote: > On Mon, Jun 6, 2016 at 7:24 PM, Brad King wrote: >> On 06/06/2016 11:39 AM, Tobias Hunger wrote: >> >>> A big chunk of Stephen's work has not even landed in my branch yet. Since >>> cmake >>> reformated all the source in the meantime it is a bit tedious to apply >>> patches >>> from his tree and I have simply not yet needed the changes as I did not >>> venture >>> where he went yet. >> See commit v3.6.0-rc1~54^2~2 (82df6deaaf). Its commit message explains >> how to rebase across the style transition. If you rebase the original >> "cmake-daemon" branch on that then we can rewrite the style to make >> cherry-picking later easier. > When I rebased my own branches across the style transiton, I found the > process quite straight forward. > I can help with the style transition if someone rebases to 82df6deaaf. > > When I tried to rebase steve's work, it was not the style transition > but changes like the following that I struggled with: > https://github.com/Kitware/CMake/commit/563bf9dd8a207e04697a92d8bcd239f52400b355 I have rebased it and force pushed my github branch (there are a few rebasing mistakes which I'll remove later). We hit issues at work that different clang-format versions have different behavior. There we have the luxury of providing particular binaries that every developer uses to get identical results and no diff confusion. I don't know how to solve that in CMake though. It could become an issue in the future when some contributors use clang-3.9 and others 3.8. Maybe clang-format binaries should be in the repo. Also, it seems that clang-format is not run by CI. The ProductBuild stuff doesn't seem to be clang formatted. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Brad, Am 07.06.2016 19:23 schrieb "Brad King" : > If it doesn't conflict with 'master' too much I can take it directly. It should not. I regularly rebase my branch on top of master. > I've cherry-picked 124f8295bee0c228b79a5cf38f0b2581be308118 and merged > to 'next' for testing: > > cmLocalGenerator: Pass configuration to GetTargetFlags > https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=fe0d2241 Great, thanks! I would love to keep my patch set as small as possible:) Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/07/2016 12:18 PM, Tobias Hunger wrote: >> When you have a few such independent refactoring changes done >> we can look at integrating them immediately to avoid holding >> them externally in your daemon topic for too long. > > Can you take that patch from github or do you want them sent here? If it doesn't conflict with 'master' too much I can take it directly. I've cherry-picked 124f8295bee0c228b79a5cf38f0b2581be308118 and merged to 'next' for testing: cmLocalGenerator: Pass configuration to GetTargetFlags https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=fe0d2241 Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Brad, sorry for the empty mail, my phone acted up:-) Am 07.06.2016 16:48 schrieb "Brad King" : > > On 06/07/2016 10:42 AM, Tobias Hunger wrote: > > On Mo, 2016-06-06 at 13:24 -0400, Brad King wrote: > >> We should be able to refactor things to share the flags computation. > >> Methods like > >> > >> cmLocalGenerator::GetTargetFlags > > > > Does this patch here make sense: > > > > https://github.com/hunger/CMake/commit/124f8295bee0c228b79a5cf38f0b2581be308118 > > > > It passes the configuration to GetTargetFlags instead of forcing it to the build > > type. Since I may end up with different configurations in the project structure, > > I think this is necessary. > > Yes. Previously that method was only called in single-configuration > generators so it could look up the CMAKE_BUILD_TYPE directly. In > multi-config generators that variable is meaningless and instead they > iterate over CMAKE_CONFIGURATION_TYPES. > > When you have a few such independent refactoring changes done > we can look at integrating them immediately to avoid holding > them externally in your daemon topic for too long. Can you take that patch from github or do you want them sent here? Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Am 07.06.2016 16:48 schrieb "Brad King" : > On 06/07/2016 10:42 AM, Tobias Hunger wrote: > > On Mo, 2016-06-06 at 13:24 -0400, Brad King wrote: > >> We should be able to refactor things to share the flags computation. > >> Methods like > >> > >> cmLocalGenerator::GetTargetFlags > > > > Does this patch here make sense: > > > > > https://github.com/hunger/CMake/commit/124f8295bee0c228b79a5cf38f0b2581be308118 > > > > It passes the configuration to GetTargetFlags instead of forcing it to > the build > > type. Since I may end up with different configurations in the project > structure, > > I think this is necessary. > > Yes. Previously that method was only called in single-configuration > generators so it could look up the CMAKE_BUILD_TYPE directly. In > multi-config generators that variable is meaningless and instead they > iterate over CMAKE_CONFIGURATION_TYPES. > > When you have a few such independent refactoring changes done > we can look at integrating them immediately to avoid holding > them externally in your daemon topic for too long. > > Thanks, > -Brad > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/07/2016 10:42 AM, Tobias Hunger wrote: > On Mo, 2016-06-06 at 13:24 -0400, Brad King wrote: >> We should be able to refactor things to share the flags computation. >> Methods like >> >> cmLocalGenerator::GetTargetFlags > > Does this patch here make sense: > > https://github.com/hunger/CMake/commit/124f8295bee0c228b79a5cf38f0b2581be308118 > > It passes the configuration to GetTargetFlags instead of forcing it to the > build > type. Since I may end up with different configurations in the project > structure, > I think this is necessary. Yes. Previously that method was only called in single-configuration generators so it could look up the CMAKE_BUILD_TYPE directly. In multi-config generators that variable is meaningless and instead they iterate over CMAKE_CONFIGURATION_TYPES. When you have a few such independent refactoring changes done we can look at integrating them immediately to avoid holding them externally in your daemon topic for too long. Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On Mo, 2016-06-06 at 13:24 -0400, Brad King wrote: > We should be able to refactor things to share the flags computation. > Methods like > > cmLocalGenerator::GetTargetFlags Does this patch here make sense: https://github.com/hunger/CMake/commit/124f8295bee0c228b79a5cf38f0b2581be308118 It passes the configuration to GetTargetFlags instead of forcing it to the build type. Since I may end up with different configurations in the project structure, I think this is necessary. Best Regards, Tobias -- Tobias Hunger, Senior Software Engineer | The Qt Company The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/07/2016 09:07 AM, Tobias Hunger wrote:
>> We should be able to refactor things to share the flags computation.
>> Methods like
>>
>> cmLocalGenerator::GetTargetFlags
>> cmLocalGenerator::GetIncludeDirectories
>>
>> are meant to be used across multiple generators.
>
> So is that the complete picture?
No, just a starting point.
Note that we have no API to get all the flags for a source because
all the generators collect target-wide and per-source flags separately.
They are not combined until the native build tool constructs the final
command line. The implementation of CMAKE_EXPORT_COMPILE_COMMANDS may
already have some code to combine flags within CMake directly though.
> You can override compile time flags on a per-file basis, based on the
> code I ran across in the ninja generator:-) Do I still need to add
> those per-source-file flags somehow?
Yes. It looks like each generator currently does its own call to
source->GetProperty("COMPILE_FLAGS")
to get the per-source flags. The daemon would have to do something
similar, possibly by factoring the logic out of the generators into
a common helper.
-Brad
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/07/2016 05:27 AM, Daniel Pfeifer wrote: > If we have a daemon mode that provides this information (as your > implementation already does), we could let ccmake and cmake-gui use > the daemon instead of linking against CMakeLib. That would be really nice. A lot of the state space in CMake's internal representation is muddled by historical evolution of the different ways we have to invoke it (command line, ccmake, cmake-gui, ctest internals, etc.). Separating the interactive modes would be a nice cleanup. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Brad! On Mon, Jun 6, 2016 at 7:24 PM, Brad King wrote: >> A big chunk of Stephen's work has not even landed in my branch yet. Since >> cmake >> reformated all the source in the meantime it is a bit tedious to apply >> patches >> from his tree and I have simply not yet needed the changes as I did not >> venture >> where he went yet. > > See commit v3.6.0-rc1~54^2~2 (82df6deaaf). Its commit message explains > how to rebase across the style transition. If you rebase the original > "cmake-daemon" branch on that then we can rewrite the style to make > cherry-picking later easier. Thanks, that might prove helpful. So far i basically picked the same sha, that worked surprisingly well. >> At this time I think I will need to duplicate a chunk of code from one of the >> generators to find the flags. Is that really necessary? If so: Which >> generator >> should I copy the code from? > > We should be able to refactor things to share the flags computation. > Methods like > > cmLocalGenerator::GetTargetFlags > cmLocalGenerator::GetIncludeDirectories > > are meant to be used across multiple generators. Due to historical > evolution in generator design not everything has been refactored to > cleanly share such infrastructure across all generators. Stephen made > a lot of progress on that front, but there is still a way to go IIRC. So is that the complete picture? You can override compile time flags on a per-file basis, based on the code I ran across in the ninja generator:-) Do I still need to add those per-source-file flags somehow? Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
Hi Daniel, On Tue, Jun 7, 2016 at 11:27 AM, Daniel Pfeifer wrote: > On Mon, Jun 6, 2016 at 5:39 PM, Tobias Hunger wrote: >> Please help to support your use-cases. > > A while ago I wrote a graphical cache editor in GTK: > https://github.com/purpleKarrot/cmake-gtk > The tool reads the cache and provides a graphical view to modify it. > It can then write the cache, run cmake, and read it back. > > I wanted to see how close we can get to ccmake or cmake-gui without > linking against CMakeLib. > > Here is what I was *unable* to achieve: > * Separating Configure from Generate (two buttons). > * Progress bar. > > Quite a short list, right? > If we have a daemon mode that provides this information (as your > implementation already does), we could let ccmake and cmake-gui use > the daemon instead of linking against CMakeLib. If the daemon can expose the right information, then you could even be able to write a generator in e.g. python;-) Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On Mon, Jun 6, 2016 at 7:24 PM, Brad King wrote: > On 06/06/2016 11:39 AM, Tobias Hunger wrote: > >> A big chunk of Stephen's work has not even landed in my branch yet. Since >> cmake >> reformated all the source in the meantime it is a bit tedious to apply >> patches >> from his tree and I have simply not yet needed the changes as I did not >> venture >> where he went yet. > > See commit v3.6.0-rc1~54^2~2 (82df6deaaf). Its commit message explains > how to rebase across the style transition. If you rebase the original > "cmake-daemon" branch on that then we can rewrite the style to make > cherry-picking later easier. When I rebased my own branches across the style transiton, I found the process quite straight forward. I can help with the style transition if someone rebases to 82df6deaaf. When I tried to rebase steve's work, it was not the style transition but changes like the following that I struggled with: https://github.com/Kitware/CMake/commit/563bf9dd8a207e04697a92d8bcd239f52400b355 cheers, Daniel -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On Mon, Jun 6, 2016 at 5:39 PM, Tobias Hunger wrote: > Please help to support your use-cases. A while ago I wrote a graphical cache editor in GTK: https://github.com/purpleKarrot/cmake-gtk The tool reads the cache and provides a graphical view to modify it. It can then write the cache, run cmake, and read it back. I wanted to see how close we can get to ccmake or cmake-gui without linking against CMakeLib. Here is what I was *unable* to achieve: * Separating Configure from Generate (two buttons). * Progress bar. Quite a short list, right? If we have a daemon mode that provides this information (as your implementation already does), we could let ccmake and cmake-gui use the daemon instead of linking against CMakeLib. cheers, Daniel > Please report what information you would like to have available from your > cmake > projects via daemon-mode. > > Patches are of course welcome at any time! > > Any help in getting any interesting information out of cmake is highly > appreciated, too. > > At this time I think I will need to duplicate a chunk of code from one of the > generators to find the flags. Is that really necessary? If so: Which generator > should I copy the code from? > > Best Regards, > Tobias > > -- > Tobias Hunger, Senior Software Engineer | The Qt Company > The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin > Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der > Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 > B > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/06/2016 05:39 PM, Tobias Hunger wrote: > Hi everybody, > > I want the cmake daemon mode Stephen Kelly proposed a while back. In my > opinion > this could become a game changer. Unfortunately Stephen told me that he did > not > have the resources to push this forward at this time. It is almost the case that you and Daniel are picking up where I left of in different areas. That's two resources :). Which is to say - I'm happy to give advice and guidance on this where I can, but I'm not currently spending my free time hacking on CMake. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon-mode
On 06/06/2016 11:39 AM, Tobias Hunger wrote: > This is why I started working on making Stephen's code merge-able and more > robust. I will also volunteer to help maintain this code going forward. Great, thanks! Hopefully I will have time to look at this in more detail soon. For now here are a few responses. > So far I copied over the daemon itself from Stephen. On top of that I added > some > nicer request/respond handling code (you get pretty similar messages from > everything, you can set a cookie in a request, which will be handed back on > response/error, there is a defined way to report a response or an error > condition), progress reporting (directly hooked up to cmake), and a couple of > basic commands to get started (reset, etc.), and some future-proofing by being > able to support several versions of the daemon protocol. Good. I like the use of libuv. I've been meaning to add the infrastructure to make libuv available as something we can use in CMake upstream so that we can port process execution away from KWSys and to libuv. The daemon mode will be a great client for libuv too. > A big chunk of Stephen's work has not even landed in my branch yet. Since > cmake > reformated all the source in the meantime it is a bit tedious to apply patches > from his tree and I have simply not yet needed the changes as I did not > venture > where he went yet. See commit v3.6.0-rc1~54^2~2 (82df6deaaf). Its commit message explains how to rebase across the style transition. If you rebase the original "cmake-daemon" branch on that then we can rewrite the style to make cherry-picking later easier. > At this time I think I will need to duplicate a chunk of code from one of the > generators to find the flags. Is that really necessary? If so: Which generator > should I copy the code from? We should be able to refactor things to share the flags computation. Methods like cmLocalGenerator::GetTargetFlags cmLocalGenerator::GetIncludeDirectories are meant to be used across multiple generators. Due to historical evolution in generator design not everything has been refactored to cleanly share such infrastructure across all generators. Stephen made a lot of progress on that front, but there is still a way to go IIRC. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
[cmake-developers] CMake daemon-mode
Hi everybody,
I want the cmake daemon mode Stephen Kelly proposed a while back. In my opinion
this could become a game changer. Unfortunately Stephen told me that he did not
have the resources to push this forward at this time.
This is why I started working on making Stephen's code merge-able and more
robust. I will also volunteer to help maintain this code going forward.
My main interest is getting project structure reported out of cmake. So this is
what I did start out with.
The cool features Stephen demo-ed in his blog
https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/
are definitely on my wish list though, but I need first class cmake project
support first -- with all the information Qt Creator can currently not get its
hands on.
So I set up a branch over on github and did some hacking:
https://github.com/hunger/CMake/commits/daemon-mode
Status:
===
So far I copied over the daemon itself from Stephen. On top of that I added some
nicer request/respond handling code (you get pretty similar messages from
everything, you can set a cookie in a request, which will be handed back on
response/error, there is a defined way to report a response or an error
condition), progress reporting (directly hooked up to cmake), and a couple of
basic commands to get started (reset, etc.), and some future-proofing by being
able to support several versions of the daemon protocol.
This infrastructure part is pretty solid by now, and has documentation and
tests.
On top of that I started to work towards loading a project and extracting the
project structure from it. This part is still pretty much in flux as I learn
where all the bits and pieces of data are stored in cmake.
This part is without proper documentation and tests at this time.
I do not consider this branch merge-worthy yet, but I would appreciate feedback
on the patch set *A LOT*, ranging from coding style issues up to hints about
things I should add to enable more use-cases, or hints on how to get to
information currently missing from the output.
A big chunk of Stephen's work has not even landed in my branch yet. Since cmake
reformated all the source in the meantime it is a bit tedious to apply patches
from his tree and I have simply not yet needed the changes as I did not venture
where he went yet. It also leaves daemon-mode as a rather separate set of
patches that is not changing core cmake components. I also hope this helps with
review at this time.
Run through:
===
Start "cmake -E daemon" (no additional parameters required). This will enable
daemon-mode and allow you to send magic text via stdin to the daemon, who will
respond with magic text on stdout. We will probably want to change that later to
a channel that does not get interleaved with cmake output, but for now it works
well enough and is really easy to test.
A typical session will currently include the following messages sent to
the cmake daemon:
[== CMake MetaMagic ==[
{"protocolVersion":{"major":0,"minor":1},"type":"handshake"}
]== CMake MetaMagic ==]
This needs to be the first thing sent as it establishes the protocol version,
right after the daemon reports the available protocol versions on start-up.
[== CMake MetaMagic ==[
{"type":"setGlobalSettings", "currentGenerator":"Ninja",
"sourceDirectory":"/home/code/src/cmake", "buildDirectory":"/tmp/cmake-build-
test"}
]== CMake MetaMagic ==]
This sets up a new build directory /tmp/cmake-build-test for the sources found
in /home/code/src/cmake, using the Ninja generator. You should be able to leave
out the currentGenerator and the sourceDirectory if the buildDirectory already
exists.
[== CMake MetaMagic ==[
{"type":"configure"}
]== CMake MetaMagic ==]
Configures the project. You can also try:
[== CMake MetaMagic ==[
{"type":"configure", "cacheArguments":["-Dsomething=else"]}
]== CMake MetaMagic ==]
to pass configure arguments along to cmake.
Next step is:
[== CMake MetaMagic ==[
{"type":"generate"}
]== CMake MetaMagic ==]
which will generate the build system in the build directory. This and configure
will produce progress output along the way.
At this point you can query the project structure:
[== CMake MetaMagic ==[
{"type":"project"}
]== CMake MetaMagic ==]
This will dump a lot of output:-) You can trim down on that a bit by limiting
target types, etc.
Additional commands supported at this time:
[== CMake MetaMagic ==[
{"type":"globalSettings"}
]== CMake MetaMagic ==]
which lists the things you can set via "setGlobalSettings".
[== CMake MetaMagic ==[
{"type":"reset"}
]== CMake MetaMagic ==]
which will reset the daemon to its default state.
Todo:
===
* Handle toolsets for generators
* Get CMakeCache via the daemon (after configure?)
* Get build system files from the daemon (after configure?)
* Get missing information on the project:
* Defines
* Compiler flags
* Information on linkage of targets
* Group similar sources to shorten the output
* S
Re: [cmake-developers] CMake daemon for user tools
On 03/25/2016 09:25 AM, Tobias Hunger wrote: > I am personally rather unsure about how to proceed. I can help make > this branch production/merge ready, but I do not want to maintain it > indefinitely after it is merged. It touches to many CMake internals > for me to be comfortable hacking on that code. Yes, this lack of interest in general in maintaining it upstream is currently a demotivator. I think we need a group of people interested in doing that. Brad/Ben, no response since I posted the blog or the code. Any word of interest in any of it? Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
Taylor Braun-Jones wrote: > On Sat, Jan 30, 2016 at 2:06 AM, Stephen Kelly > wrote: >> I've just pushed the daemon code here: >> >> https://github.com/steveire/cmake/tree/cmake-daemon >> >> The Kate plugin should soon appear here I think: >> >> https://quickgit.kde.org/?p=scratch%2Fskelly%2Fcmakekate.git > > Thanks for sharing, Stephen! Do you plan to also share the standalone > Qt-based editor/client as well (cmake-browser)? That would be > immediately useful to me and allow me to more easily test drive all > this work that you've done. I've deliberately not pushed that yet because it would put the wrong focus on the conversation: * It doesn't need a test drive because I know it is limited by the daemon * The focus needs to be on the implementation of the daemon first to make that work better. That can be done by anyone determined enough to paste some things into the stdin. For more see http://article.gmane.org/gmane.comp.programming.tools.cmake.devel/15740 http://article.gmane.org/gmane.comp.programming.tools.cmake.devel/15741 Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
On Sat, Jan 30, 2016 at 2:06 AM, Stephen Kelly wrote: > I've just pushed the daemon code here: > > https://github.com/steveire/cmake/tree/cmake-daemon > > The Kate plugin should soon appear here I think: > > https://quickgit.kde.org/?p=scratch%2Fskelly%2Fcmakekate.git Thanks for sharing, Stephen! Do you plan to also share the standalone Qt-based editor/client as well (cmake-browser)? That would be immediately useful to me and allow me to more easily test drive all this work that you've done. Cheers, Taylor -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
Nice work! I would love to help but I'm already overloaded with tons of work for now. :/ On 14 February 2016 at 13:47, Tobias Hunger wrote: > Hi Klaim, > > I spent the WE to improve the unit tests in my fork of Stephen's > cmake-daemon (which got broken when I added some basic protocol features > earlier in the same fork). > > I could use some help in this task that does not involve C++ coding:-) > > Best Regards, > Tobias > Am 14.02.2016 12:58 schrieb "Klaim - Joël Lamotte" : > >> I'm surprised too that there isn't more feedback around here. >> This seems to be a marjor potential improvement and definitely necessary >> if CMake need to be usable in the coming years. >> >> On 10 February 2016 at 09:42, Tobias Hunger >> wrote: >> >>> Hey, I did send a MR on github:-) >>> >>> But I agree, I had also hoped for more active participation. >>> >>> Best Regards, >>> Tobias >>> >>> On Wed, Feb 10, 2016 at 12:07 AM, Stephen Kelly >>> wrote: >>> > Stephen Kelly wrote: >>> > >>> >> Hi, >>> >> >>> >> I just made a blog and video about the advanced features and >>> possibilities >>> >> that a daemon mode for CMake can bring: >>> >> >>> >> >>> https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ >>> >> >>> > >>> > >>> > The lack of response on the list to any of this is quite disappointing. >>> > >>> > -- >>> > >>> > Powered by www.kitware.com >>> > >>> > Please keep messages on-topic and check the CMake FAQ at: >>> http://www.cmake.org/Wiki/CMake_FAQ >>> > >>> > Kitware offers various services to support the CMake community. For >>> more information on each offering, please visit: >>> > >>> > CMake Support: http://cmake.org/cmake/help/support.html >>> > CMake Consulting: http://cmake.org/cmake/help/consulting.html >>> > CMake Training Courses: http://cmake.org/cmake/help/training.html >>> > >>> > Visit other Kitware open-source projects at >>> http://www.kitware.com/opensource/opensource.html >>> > >>> > Follow this link to subscribe/unsubscribe: >>> > http://public.kitware.com/mailman/listinfo/cmake-developers >>> -- >>> >>> Powered by www.kitware.com >>> >>> Please keep messages on-topic and check the CMake FAQ at: >>> http://www.cmake.org/Wiki/CMake_FAQ >>> >>> Kitware offers various services to support the CMake community. For more >>> information on each offering, please visit: >>> >>> CMake Support: http://cmake.org/cmake/help/support.html >>> CMake Consulting: http://cmake.org/cmake/help/consulting.html >>> CMake Training Courses: http://cmake.org/cmake/help/training.html >>> >>> Visit other Kitware open-source projects at >>> http://www.kitware.com/opensource/opensource.html >>> >>> Follow this link to subscribe/unsubscribe: >>> http://public.kitware.com/mailman/listinfo/cmake-developers >>> >> >> -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
Hi Klaim, I spent the WE to improve the unit tests in my fork of Stephen's cmake-daemon (which got broken when I added some basic protocol features earlier in the same fork). I could use some help in this task that does not involve C++ coding:-) Best Regards, Tobias Am 14.02.2016 12:58 schrieb "Klaim - Joël Lamotte" : > I'm surprised too that there isn't more feedback around here. > This seems to be a marjor potential improvement and definitely necessary > if CMake need to be usable in the coming years. > > On 10 February 2016 at 09:42, Tobias Hunger > wrote: > >> Hey, I did send a MR on github:-) >> >> But I agree, I had also hoped for more active participation. >> >> Best Regards, >> Tobias >> >> On Wed, Feb 10, 2016 at 12:07 AM, Stephen Kelly >> wrote: >> > Stephen Kelly wrote: >> > >> >> Hi, >> >> >> >> I just made a blog and video about the advanced features and >> possibilities >> >> that a daemon mode for CMake can bring: >> >> >> >> >> https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ >> >> >> > >> > >> > The lack of response on the list to any of this is quite disappointing. >> > >> > -- >> > >> > Powered by www.kitware.com >> > >> > Please keep messages on-topic and check the CMake FAQ at: >> http://www.cmake.org/Wiki/CMake_FAQ >> > >> > Kitware offers various services to support the CMake community. For >> more information on each offering, please visit: >> > >> > CMake Support: http://cmake.org/cmake/help/support.html >> > CMake Consulting: http://cmake.org/cmake/help/consulting.html >> > CMake Training Courses: http://cmake.org/cmake/help/training.html >> > >> > Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> > >> > Follow this link to subscribe/unsubscribe: >> > http://public.kitware.com/mailman/listinfo/cmake-developers >> -- >> >> Powered by www.kitware.com >> >> Please keep messages on-topic and check the CMake FAQ at: >> http://www.cmake.org/Wiki/CMake_FAQ >> >> Kitware offers various services to support the CMake community. For more >> information on each offering, please visit: >> >> CMake Support: http://cmake.org/cmake/help/support.html >> CMake Consulting: http://cmake.org/cmake/help/consulting.html >> CMake Training Courses: http://cmake.org/cmake/help/training.html >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Follow this link to subscribe/unsubscribe: >> http://public.kitware.com/mailman/listinfo/cmake-developers >> > > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
I'm surprised too that there isn't more feedback around here. This seems to be a marjor potential improvement and definitely necessary if CMake need to be usable in the coming years. On 10 February 2016 at 09:42, Tobias Hunger wrote: > Hey, I did send a MR on github:-) > > But I agree, I had also hoped for more active participation. > > Best Regards, > Tobias > > On Wed, Feb 10, 2016 at 12:07 AM, Stephen Kelly > wrote: > > Stephen Kelly wrote: > > > >> Hi, > >> > >> I just made a blog and video about the advanced features and > possibilities > >> that a daemon mode for CMake can bring: > >> > >> https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ > >> > > > > > > The lack of response on the list to any of this is quite disappointing. > > > > -- > > > > Powered by www.kitware.com > > > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > > > CMake Support: http://cmake.org/cmake/help/support.html > > CMake Consulting: http://cmake.org/cmake/help/consulting.html > > CMake Training Courses: http://cmake.org/cmake/help/training.html > > > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/cmake-developers > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon
Hi Stephen,
thanks for that information! Now I can finally fix the tests again
(and add a couple for the error cases
my branch handles;-).
I did play some more and implemented support for consistent progress
reporting and multiple protocol versions now in my branch at
https://github.com/hunger/CMake/tree/cmake-daemon
This unfortunately resulted in the interaction with the daemon changing a bit:-(
Upon start the daemon now sends:
{"supportedProtocolVersions":[{"major":0,"minor":1}],"type":"hello"}
You need to reply with something along the lines off:
{"type":"handshake", "protocolVersion": { "major": 0 }}
Daemon will respond with:
{"cookie":"","inResponseTo":"handshake","type":"reply"}
and listen for commands.
"version" will work at this point, but you need to do:
{"type":"initialize", "buildDirectory":"/some/build/dir"}
to trigger the actual parsing of data.
Afterwards all the usual commands are available as before.
I did not yet update the tests with this change, so I did not create a MR yet.
I also want to do some small (but intrusive) changes to how
cmServerResponse objects are created.
Best Regards,
Tobias
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon
Tobias Hunger wrote: > Hi Stephen, > > stupid question: How do I run the server-mode autotests? > > I just implemented a unified way to do progress reporting and want to > make sure that did not break too much. Running > Tests/Server/daemon-test.py manually works, but how do I run all those > tests configured in the CMakeLists.txt? The builddirectory does not > contain a Tests/Server directory to run make test in:-/ After building cmake, run ctest -R Server in the build directory Alternatively, cd to Tests/Server in the source directory and mkdir build cd build cmake .. The tests are run at cmake time. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon
Hi Stephen, stupid question: How do I run the server-mode autotests? I just implemented a unified way to do progress reporting and want to make sure that did not break too much. Running Tests/Server/daemon-test.py manually works, but how do I run all those tests configured in the CMakeLists.txt? The builddirectory does not contain a Tests/Server directory to run make test in:-/ Best Regards, Tobias On Wed, Feb 10, 2016 at 12:44 AM, Stephen Kelly wrote: > Stephen Kelly wrote: > >> Tamás Kenéz wrote: >> >>> That's great and really does open a new world for IDEs! >> >> Thanks! Let's see if the interest grows. >> >> I've just pushed the daemon code here: >> >> https://github.com/steveire/cmake/tree/cmake-daemon > > Tobias made a pull request there. Rather than review it there, I will review > it here for visibility. > > https://github.com/steveire/CMake/pull/2 > > The branch is quite it hard to review, or even to see the particular > changes, due to large commits and diff noise. If the Daemon reaches a level > of completeness that it could be upstreamed (See > > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15740 > > ) then these commits (including all of my commits on the branch) would have > to be rewritten, split, made reviewable etc making heavy use of `rebase -i`. > > In a way, we don't have to do that now, but I'm also not very enthusiastic > about making the `cmake-daemon` branch commits unreadable. I would add your > commits to the branch if they we split and in the appropriate place (eg, > with the cmServerProtocol0_1 change early in the cmake-daemon branch). > > The changes in your branch are good and useful to more than just QtCreator. > > Things that I like in your branch: > > * Explicit cmServerRequest and cmServerResponse APIs, which enforce the type > and cookie consistency. > * Returning cmServerResponse objects from the cmServerProtocol instead of > invoking the server from the cmServerProtocol. > * A way to version the protocol in a future-proof way with C++ classes. > * Implementation of daemon and protocol error messaging infrastructure. > (Reporting errors from cmake code requires other refactoring: > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15607/focus=15636) > > So I think that is progress! > > Thanks, > > Steve. > > > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon
Hi Stephen, I will then continue down that road. Next steps are a way to consistently report progress as well as being able to have several protocol versions at the same time and have a client pick one. I will just put them atop my current branch, as that is going to need basically the same amount of retro-active integration into similar places as the current changes. I am not keen on doing that twice:-) Best Regards, Tobias On Wed, Feb 10, 2016 at 12:44 AM, Stephen Kelly wrote: > Stephen Kelly wrote: > >> Tamás Kenéz wrote: >> >>> That's great and really does open a new world for IDEs! >> >> Thanks! Let's see if the interest grows. >> >> I've just pushed the daemon code here: >> >> https://github.com/steveire/cmake/tree/cmake-daemon > > Tobias made a pull request there. Rather than review it there, I will review > it here for visibility. > > https://github.com/steveire/CMake/pull/2 > > The branch is quite it hard to review, or even to see the particular > changes, due to large commits and diff noise. If the Daemon reaches a level > of completeness that it could be upstreamed (See > > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15740 > > ) then these commits (including all of my commits on the branch) would have > to be rewritten, split, made reviewable etc making heavy use of `rebase -i`. > > In a way, we don't have to do that now, but I'm also not very enthusiastic > about making the `cmake-daemon` branch commits unreadable. I would add your > commits to the branch if they we split and in the appropriate place (eg, > with the cmServerProtocol0_1 change early in the cmake-daemon branch). > > The changes in your branch are good and useful to more than just QtCreator. > > Things that I like in your branch: > > * Explicit cmServerRequest and cmServerResponse APIs, which enforce the type > and cookie consistency. > * Returning cmServerResponse objects from the cmServerProtocol instead of > invoking the server from the cmServerProtocol. > * A way to version the protocol in a future-proof way with C++ classes. > * Implementation of daemon and protocol error messaging infrastructure. > (Reporting errors from cmake code requires other refactoring: > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15607/focus=15636) > > So I think that is progress! > > Thanks, > > Steve. > > > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
Hey, I did send a MR on github:-) But I agree, I had also hoped for more active participation. Best Regards, Tobias On Wed, Feb 10, 2016 at 12:07 AM, Stephen Kelly wrote: > Stephen Kelly wrote: > >> Hi, >> >> I just made a blog and video about the advanced features and possibilities >> that a daemon mode for CMake can bring: >> >> https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ >> > > > The lack of response on the list to any of this is quite disappointing. > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon
Stephen Kelly wrote: > Tamás Kenéz wrote: > >> That's great and really does open a new world for IDEs! > > Thanks! Let's see if the interest grows. > > I've just pushed the daemon code here: > > https://github.com/steveire/cmake/tree/cmake-daemon Tobias made a pull request there. Rather than review it there, I will review it here for visibility. https://github.com/steveire/CMake/pull/2 The branch is quite it hard to review, or even to see the particular changes, due to large commits and diff noise. If the Daemon reaches a level of completeness that it could be upstreamed (See http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15740 ) then these commits (including all of my commits on the branch) would have to be rewritten, split, made reviewable etc making heavy use of `rebase -i`. In a way, we don't have to do that now, but I'm also not very enthusiastic about making the `cmake-daemon` branch commits unreadable. I would add your commits to the branch if they we split and in the appropriate place (eg, with the cmServerProtocol0_1 change early in the cmake-daemon branch). The changes in your branch are good and useful to more than just QtCreator. Things that I like in your branch: * Explicit cmServerRequest and cmServerResponse APIs, which enforce the type and cookie consistency. * Returning cmServerResponse objects from the cmServerProtocol instead of invoking the server from the cmServerProtocol. * A way to version the protocol in a future-proof way with C++ classes. * Implementation of daemon and protocol error messaging infrastructure. (Reporting errors from cmake code requires other refactoring: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15607/focus=15636) So I think that is progress! Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
Stephen Kelly wrote: > Hi, > > I just made a blog and video about the advanced features and possibilities > that a daemon mode for CMake can bring: > > https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ > The lack of response on the list to any of this is quite disappointing. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
Tamás Kenéz wrote: > That's great and really does open a new world for IDEs! Thanks! Let's see if the interest grows. I've just pushed the daemon code here: https://github.com/steveire/cmake/tree/cmake-daemon The Kate plugin should soon appear here I think: https://quickgit.kde.org/?p=scratch%2Fskelly%2Fcmakekate.git Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On 26-Jan-16 12:14 AM, Stephen Kelly wrote: Vladimir Prus wrote: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004 Hi Stephen, this sounds very much like what I has proposed for Boost.Build, see: http://vladimirprus.com/talks/boosting-cdt-builds/ Yes, I'm familiar with that. It is listed as prior art in the gmane link you quoted :) :-) I am scheduled to give a talk at FOSDEM about the feature and how user tools can interact with it: https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code ... Is anyone interested enough in this potential cmake feature to join the development effort? You can guess I'm not very interested in cmake, but maybe we can discuss the protocol, so that an IDE can work with different build systems? Think we can have a chat at FOSDEM? Yep, sounds good to me. I don't have a schedule decided yet for which talks I'll attend, but I should be reachable by email to arrange to meet. I assume you've seen https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ Yes, that is quite impressive. I don't have a schedule yet either, but I am going to attend your talk, so we can sync there. -- Vladimir Prus http://vladimirprus.com -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
Vladimir Prus wrote: > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004 > > Hi Stephen, > > this sounds very much like what I has proposed for Boost.Build, see: > > http://vladimirprus.com/talks/boosting-cdt-builds/ Yes, I'm familiar with that. It is listed as prior art in the gmane link you quoted :) >> I am scheduled to give a talk at FOSDEM about the feature and how user >> tools can interact with it: >> >> https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code > ... >> Is anyone interested enough in this potential cmake feature to >> join the development effort? > > You can guess I'm not very interested in cmake, but maybe we can discuss > the protocol, so that an IDE can work with different build systems? Think > we can have a chat at FOSDEM? Yep, sounds good to me. I don't have a schedule decided yet for which talks I'll attend, but I should be reachable by email to arrange to meet. I assume you've seen https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
> -Original Message- > From: Milian Wolff [mailto:m...@milianw.de] > Sent: Saturday, January 23, 2016 15:41 > To: cmake-developers@cmake.org > Cc: James Johnston > Subject: Re: [cmake-developers] CMake daemon for user tools > > You are aware that modern std::string is SSO'ed? I'm running on such a > system. > Another reason why you should not reinvent the wheel and keep relying on > the STL wherever possible. > > Qt has such a class, it's called QVarLengthArray, and I've also been able to > apply it in multiple occasions to good effect. That said, when you look at Yeah, but std::string is platform dependent, and the size of the buffer is also platform dependent. Maybe it tends to be optimal for CMake. Then again, maybe a larger buffer is needed. I don't know. The flexible option would be something that does exactly like QVarLengthArray. Different variables might have different optimal sizes. Some sample small strings for gcc/clang/VC++: http://stackoverflow.com/a/28003328/562766 Note that none of them are large enough to store an absolute path, which are maybe common (???) in CMake. Also there's a fair bit of variation; if CMake wants consistent performance in a section of code across compilers, it would need a way to explicitly specify the small string size. For example, some are large enough to store typical target sizes - and some maybe are not. There is also boost::container::small_vector in addition to QVarLengthArray: http://www.boost.org/doc/libs/1_60_0/doc/html/boost/container/small_vector.h tml > Just run cmake (or the daemon) through a profiler and check the results. > Doing so for the daemon (built with RelWithDebInfo) on the LLVM build dir > and recording it with `perf --call-graph lbr` I get these hotspots when looking > at the results with `perf report -g graph --no-children`: > > +8.67% cmake cmake[.] > cmGlobalGenerator::FindGeneratorTargetImpl > +4.21% cmake libc-2.22.so [.] _int_malloc > +2.67% cmake cmake[.] cmCommandArgument_yylex > +2.09% cmake libc-2.22.so [.] _int_free > +2.06% cmake libc-2.22.so [.] __memcmp_sse4_1 > +1.84% cmake libc-2.22.so [.] malloc > > This already shows you that you can gain a lot by reducing the number of > allocations done. Heaptrack is a good tool for that. Next question would be: who is calling malloc? Or rather, what % of callers are std::string, std::vector, other STL classes vs custom data structures? Next question would be: what is the size of those mallocs, for each caller? (Sorry I don't currently have an environment set up with a profiler to test this myself at the moment.) > Similarly, someone should > investigate cmGlobalGenerator::FindGeneratorTargetImpl. That does a lot of > string comparisons to find targets from my quick glance, so indeed could be > sped up with a smarter string class. > > But potentially you could also get a much quicker lookup by storing a hash > map of target name to cmGeneratorTarget. Indeed; there has got to be a way to reduce the complexity of that function in number of targets compared, if not the low-level string comparison itself as well. For example, if target names are short-ish, the string class has large enough SSO, and the underlying string class made use of vector CPU instructions for comparison, there is probably very little to be gained without such a hash map. (On the other hand, if some of the previous assumptions are not true on some common CMake platforms) > Seems like there's more than enough areas one could (and should) optimize > CMake. Indeed. Another idea - probably unrelated to the string allocations issue, but still - that came to mind: what if link-time code generation/optimization is turned on? IIRC this is not default in CMake. Maybe CMake is sufficiently well-organized (e.g. small functions implementations moved to header files) such that what needs to be inlined across units, is already being inlined. Then again, maybe it's not. I've seen other projects rely on this feature to keep clean organization by keeping implementations in .CPP files without sacrificing performance, and when you turn off LTCG performance takes a major hit... Also IIRC there are still a few optimizations that are turned off when CMake is built with RelWithDebInfo instead of Release. I forget the exact specifics at the moment but e.g. on Visual C++ when you ask it to turn on debug symbols, it will change the default values of some optimization flags. So a cursory examination of the flags wouldn't reveal all cases. However, one of my bigger performance gripes, being a primarily Windows developer right now, is the process creation overhead, especially during configuration. I think that
Re: [cmake-developers] CMake daemon for user tools
On 10-Jan-16 2:10 PM, Stephen Kelly wrote: Hello, I've been working on adding a daemon mode for cmake to provide information to user tools - such as IDEs - about the buildsystem. Following the discussion about providing metadata for IDEs to consume I proposed creating a long-running process which would provide a protocol to access information about the buildsystem, and about the content of the cmake files themselves: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004 Hi Stephen, this sounds very much like what I has proposed for Boost.Build, see: http://vladimirprus.com/talks/boosting-cdt-builds/ I am scheduled to give a talk at FOSDEM about the feature and how user tools can interact with it: https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code ... Is anyone interested enough in this potential cmake feature to join the development effort? You can guess I'm not very interested in cmake, but maybe we can discuss the protocol, so that an IDE can work with different build systems? Think we can have a chat at FOSDEM? -- Vladimir Prus http://vladimirprus.com -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake Daemon blog
That's great and really does open a new world for IDEs! Tamas On Sun, Jan 24, 2016 at 11:44 PM, Stephen Kelly wrote: > Hi, > > I just made a blog and video about the advanced features and possibilities > that a daemon mode for CMake can bring: > > https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ > > I'll make the code available by the end of the week. I still want to clean > it up a little bit first :). > > Thanks, > > Steve. > > > -- > > Powered by www.kitware.com > > Please keep messages on-topic and check the CMake FAQ at: > http://www.cmake.org/Wiki/CMake_FAQ > > Kitware offers various services to support the CMake community. For more > information on each offering, please visit: > > CMake Support: http://cmake.org/cmake/help/support.html > CMake Consulting: http://cmake.org/cmake/help/consulting.html > CMake Training Courses: http://cmake.org/cmake/help/training.html > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/cmake-developers > -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
[cmake-developers] CMake Daemon blog
Hi, I just made a blog and video about the advanced features and possibilities that a daemon mode for CMake can bring: https://steveire.wordpress.com/2016/01/24/cmake-daemon-for-user-tools/ I'll make the code available by the end of the week. I still want to clean it up a little bit first :). Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On Freitag, 22. Januar 2016 17:55:46 CET James Johnston wrote: > > -Original Message- > > From: cmake-developers [mailto:cmake-developers-boun...@cmake.org] > > On Behalf Of Milian Wolff > > Sent: Thursday, January 21, 2016 22:31 > > To: Daniel Pfeifer > > Cc: CMake Developers; Stephen Kelly > > Subject: Re: [cmake-developers] CMake daemon for user tools > > > > > What do you think about string interning? I started a cmString class > > > [2] that stores short strings inplace and puts long strings into a > > > pool. Copies never allocate extra memory and equality comparison is > > > always O(1). > > > In addition to `operator<` (which is lexicographical) there is a O(1) > > > comparison that can be used where lexicographical ordering is not > > > required (ie. for lookup tables). > > > > > > [1] https://en.wikipedia.org/wiki/String_interning > > > [2] https://github.com/purpleKarrot/CMake/commits/string-pool > > > > Imo, you should replace this custom code by a boost::flyweight of > > std::string. > > > That said, I think this can have a significant impact on the memory > > footprint > > > of CMake, considering how heavy it relies on strings internally. But it > > also > > > seems to mutate strings a lot. I've seen places e.g. where a list of > > compile- > > > time known identifiers is prepended with "CMAKE_" at runtime. This is slow > > with normal strings (reallocations), but will also be slow with a > > flyweight or > > > interning, possibly even leading to the pollution of the internal pool > > with > > > temporary strings. > > > > Did you measure any impact on both, runtime speed and memory footprint > > yet? > > I was wondering the same. I would guess maybe the biggest impact would be > the inplace storage of strings for small sized strings. But to know the > inplace buffer size would probably require some profiling and measurement of > string sizes... otherwise it is just a wild guess... You are aware that modern std::string is SSO'ed? I'm running on such a system. Another reason why you should not reinvent the wheel and keep relying on the STL wherever possible. > Maybe for testing, you can swap out the string header file on your system > with one that logs allocations/string sizes, and perhaps also profiles the > time it takes to make each allocation? The data recorded by heaptrack would allow such an analysis post-mortem without modification of any header. Someone just needs to write such an analysis step... > The interesting question is: could inplace storage be used for 95% of the > cases such that fussing with string interning becomes unnecessary > complexity? If so, then you mentioned equality comparison as another issue: > the interesting question there is how much time is spent on allocations vs > comparisons... Just run cmake (or the daemon) through a profiler and check the results. Doing so for the daemon (built with RelWithDebInfo) on the LLVM build dir and recording it with `perf --call-graph lbr` I get these hotspots when looking at the results with `perf report -g graph --no-children`: +8.67% cmake cmake[.] cmGlobalGenerator::FindGeneratorTargetImpl +4.21% cmake libc-2.22.so [.] _int_malloc +2.67% cmake cmake[.] cmCommandArgument_yylex +2.09% cmake libc-2.22.so [.] _int_free +2.06% cmake libc-2.22.so [.] __memcmp_sse4_1 +1.84% cmake libc-2.22.so [.] malloc This already shows you that you can gain a lot by reducing the number of allocations done. Heaptrack is a good tool for that. Similarly, someone should investigate cmGlobalGenerator::FindGeneratorTargetImpl. That does a lot of string comparisons to find targets from my quick glance, so indeed could be sped up with a smarter string class. But potentially you could also get a much quicker lookup by storing a hash map of target name to cmGeneratorTarget. > In another application I worked on, I was able to get a big improvement in > performance by replacing usage of std::vector in one place with a custom > vector that stack-allocated the first 10 items (i.e. fixed-size C array as a > member variable of the class), and then reverted to a regular vector after > that. But to pick the number "10" required some pr
Re: [cmake-developers] CMake daemon for user tools
> -Original Message- > From: cmake-developers [mailto:cmake-developers-boun...@cmake.org] > On Behalf Of Milian Wolff > Sent: Thursday, January 21, 2016 22:31 > To: Daniel Pfeifer > Cc: CMake Developers; Stephen Kelly > Subject: Re: [cmake-developers] CMake daemon for user tools > > > What do you think about string interning? I started a cmString class > > [2] that stores short strings inplace and puts long strings into a > > pool. Copies never allocate extra memory and equality comparison is > > always O(1). > > In addition to `operator<` (which is lexicographical) there is a O(1) > > comparison that can be used where lexicographical ordering is not > > required (ie. for lookup tables). > > > > [1] https://en.wikipedia.org/wiki/String_interning > > [2] https://github.com/purpleKarrot/CMake/commits/string-pool > > Imo, you should replace this custom code by a boost::flyweight of std::string. > That said, I think this can have a significant impact on the memory footprint > of CMake, considering how heavy it relies on strings internally. But it also > seems to mutate strings a lot. I've seen places e.g. where a list of compile- > time known identifiers is prepended with "CMAKE_" at runtime. This is slow > with normal strings (reallocations), but will also be slow with a flyweight or > interning, possibly even leading to the pollution of the internal pool with > temporary strings. > > Did you measure any impact on both, runtime speed and memory footprint > yet? I was wondering the same. I would guess maybe the biggest impact would be the inplace storage of strings for small sized strings. But to know the inplace buffer size would probably require some profiling and measurement of string sizes... otherwise it is just a wild guess... Maybe for testing, you can swap out the string header file on your system with one that logs allocations/string sizes, and perhaps also profiles the time it takes to make each allocation? The interesting question is: could inplace storage be used for 95% of the cases such that fussing with string interning becomes unnecessary complexity? If so, then you mentioned equality comparison as another issue: the interesting question there is how much time is spent on allocations vs comparisons... In another application I worked on, I was able to get a big improvement in performance by replacing usage of std::vector in one place with a custom vector that stack-allocated the first 10 items (i.e. fixed-size C array as a member variable of the class), and then reverted to a regular vector after that. But to pick the number "10" required some profiling/measurement. The remaining use of the heap was so negligible as to not be worth improving. Best regards, James Johnston -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On Donnerstag, 21. Januar 2016 22:03:23 CET Daniel Pfeifer wrote: > On Wed, Jan 20, 2016 at 10:03 PM, Stephen Kelly wrote: > > Milian Wolff wrote: > >>> I'm concerned that the memory usage of a daemon implementing the > >>> proposed > >>> capabilities may be too large to be practical (at least without a major > >>> redesign of certain structures that tend to duplicate substrings, or > >>> some kind of out-of-core approach). > >> > >> This sounds like optimizations to that daemon will benefit CMake. > > > > FYI I merged a reduce-allocations branch to next for testing. It came from > > > > Milian here: > > https://github.com/steveire/CMake/pull/1 > > Very nice! It is impressive what kind of an impact a simple 'const&' > can have. Also, nice work identifying the hotspots. > > > Milian also mentioned the possibility of using something like a sqlite > > database (probably what you meant by out-of-core above) for definitions > > for > > querying by the daemon. > > > > I also mentioned some possibile optimization possibilities, such as > > removing the closure of definitions created for all directories. I wrote > > a branch which does that some months ago, but it resulted in a slow down. > > I'll see if I can rebase the commit to master and push it to github, > > together with a patch for avoiding computing the hash multiple times in > > cmDefinitions. > > What do you think about string interning? I started a cmString class > [2] that stores short strings inplace and puts long strings into a > pool. Copies never allocate extra memory and equality comparison is > always O(1). > In addition to `operator<` (which is lexicographical) there is a O(1) > comparison that can be used where lexicographical ordering is not > required (ie. for lookup tables). > > [1] https://en.wikipedia.org/wiki/String_interning > [2] https://github.com/purpleKarrot/CMake/commits/string-pool Imo, you should replace this custom code by a boost::flyweight of std::string. That said, I think this can have a significant impact on the memory footprint of CMake, considering how heavy it relies on strings internally. But it also seems to mutate strings a lot. I've seen places e.g. where a list of compile- time known identifiers is prepended with "CMAKE_" at runtime. This is slow with normal strings (reallocations), but will also be slow with a flyweight or interning, possibly even leading to the pollution of the internal pool with temporary strings. Did you measure any impact on both, runtime speed and memory footprint yet? -- Milian Wolff m...@milianw.de http://milianw.de signature.asc Description: This is a digitally signed message part. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On Wed, Jan 20, 2016 at 10:03 PM, Stephen Kelly wrote: > Milian Wolff wrote: > >>> I'm concerned that the memory usage of a daemon implementing the proposed >>> capabilities may be too large to be practical (at least without a major >>> redesign of certain structures that tend to duplicate substrings, or >>> some kind of out-of-core approach). >> >> This sounds like optimizations to that daemon will benefit CMake. > > FYI I merged a reduce-allocations branch to next for testing. It came from > Milian here: > > https://github.com/steveire/CMake/pull/1 Very nice! It is impressive what kind of an impact a simple 'const&' can have. Also, nice work identifying the hotspots. > Milian also mentioned the possibility of using something like a sqlite > database (probably what you meant by out-of-core above) for definitions for > querying by the daemon. > > I also mentioned some possibile optimization possibilities, such as removing > the closure of definitions created for all directories. I wrote a branch > which does that some months ago, but it resulted in a slow down. I'll see if > I can rebase the commit to master and push it to github, together with a > patch for avoiding computing the hash multiple times in cmDefinitions. What do you think about string interning? I started a cmString class [2] that stores short strings inplace and puts long strings into a pool. Copies never allocate extra memory and equality comparison is always O(1). In addition to `operator<` (which is lexicographical) there is a O(1) comparison that can be used where lexicographical ordering is not required (ie. for lookup tables). [1] https://en.wikipedia.org/wiki/String_interning [2] https://github.com/purpleKarrot/CMake/commits/string-pool cheers, Daniel -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On 01/20/2016 04:03 PM, Stephen Kelly wrote: > FYI I merged a reduce-allocations branch to next for testing. It came from > Milian here: > > https://github.com/steveire/CMake/pull/1 Very nice! Thanks, -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
Milian Wolff wrote: >> I'm concerned that the memory usage of a daemon implementing the proposed >> capabilities may be too large to be practical (at least without a major >> redesign of certain structures that tend to duplicate substrings, or >> some kind of out-of-core approach). > > This sounds like optimizations to that daemon will benefit CMake. FYI I merged a reduce-allocations branch to next for testing. It came from Milian here: https://github.com/steveire/CMake/pull/1 Milian also mentioned the possibility of using something like a sqlite database (probably what you meant by out-of-core above) for definitions for querying by the daemon. I also mentioned some possibile optimization possibilities, such as removing the closure of definitions created for all directories. I wrote a branch which does that some months ago, but it resulted in a slow down. I'll see if I can rebase the commit to master and push it to github, together with a patch for avoiding computing the hash multiple times in cmDefinitions. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] cmake daemon mode protocol
Hi Stephen, I think what I suggested so far is covered by the first way of interaction covered in http://thread.gmane.org/gmane.comp.lib.qt.creator/11794/focus=15411 . I am just proposing something that is convenient for me to use. On Sat, Jan 16, 2016 at 12:26 PM, Stephen Kelly wrote: >>> For this case, I suggest that if the user tries to 'open the source >>> directory', you would use QTemporaryDir to build in a temporary location >> and >>> run the daemon there. I believe clion does something equivalent to that. >>> Is that viable? I suppose you are suggesting that cmake do that instead >>> of leaving it for clients to do? >> >> It is doable. I just do not see why this should be necessary. > > I'm aiming to first design as much as possible of a 'minimal viable > protocol' as a first step. Given the above I think your ideas of running the > daemon with no build dir are not 'minimal' and I'm convinced they're not > viable. I am thinking about the simplest possible use-case I can think of right now: User gets sources from a project somewhere and wants to build those. What is entirely unclear to me with your current implementation is how do I get from "sources are checked out in $DIR" to "I have a $BUILD that I can use to start the daemon"? Then I also do not see how I can support a user switching between two build directories. Do I need to close down one daemon and start another? > Someone would have to prototype the ideas, but I don't think that's a good > use of time or energy right now (mostly because I think they're not viable). > > I recommend focussing on the tasks in my OP: > > http://thread.gmane.org/gmane.comp.lib.qt.creator/11794 That list is so full of cmake internals that it is basically meaningless to me. Sorry. >>> Richer information about some semantics like 'task' and 'busy state' >>> could also be provided in a similar way I expect, assuming those could be >> defined. >> >> This is some basic functionality that we should get right soon, as this >> can influence how long-running tasks need to be designed. > > Yes. Perhaps you can expand on what 'tasks' and 'busy state' you have in > mind. Particularly if you can relate them to what is already in my github > branch. "task" is just some Id to identify what the daemon is currently doing. "state" is just some indication of whether the daemon thinks it is busy with something or not. In "idle" state it should accept more requests, in "busy" state it will just queue them or maybe even ignore them. >> So "cmake --build" is out of scope for the daemon? > > I listed three ways cmake and an IDE could interface here: > > http://thread.gmane.org/gmane.comp.lib.qt.creator/11794/focus=15411 > > 'triggering the build' would be a fourth. I don't think it needs to be part > of the initial discussion of the design. > > Let's try to keep scope small so that we can get on common ground. So let's forget about all the interactive stuff for now. That is IMHO step 2, after the basic protocol is validated by implementing bare bones project information:-) >> That is why I brought up progress information so early: They make things >> complicated by there suddenly being several responses to a request and you >> need a way to identify those. At that point simpler solutions tend to blow >> up:-) > > Yes. > > BTW: I don't expect to get any part of the design of the protocol 'right' on > the first iteration. I think we need to start everything that will need to > be in the protocol, then iterate. Start high level ignoring details, and > fill in the details as we iterate and can encode needs in unit tests. I agree and do not expect anything different. Basic protocol functionality from my point of view needs to include: * Being able to detect the daemon or IDE becoming unresponsive * Being able to find which responses are the result of which requests * Being able to reliably find what kind of object I got as request or response >From the point of view of an IDE: Being able to have progress information is not a high level detail to be filled in later. Best Regards, Tobias -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] cmake daemon mode protocol
Stephen Kelly wrote: > I recommend focussing on the tasks in my OP: > > http://thread.gmane.org/gmane.comp.lib.qt.creator/11794 To be more clear: The goal I have is to enable debugging, introspection of the buildsystem and the state during execution, code completion etc. Your goal seems to be completely orthogonal to that and requires an entirely different set of steps to achieve (perhaps the first being 'demonstrate viability'). I don't think I can help with your goal. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] cmake daemon mode protocol
Tobias Hunger wrote: >> For this case, I suggest that if the user tries to 'open the source >> directory', you would use QTemporaryDir to build in a temporary location > and >> run the daemon there. I believe clion does something equivalent to that. >> Is that viable? I suppose you are suggesting that cmake do that instead >> of leaving it for clients to do? > > It is doable. I just do not see why this should be necessary. I'm aiming to first design as much as possible of a 'minimal viable protocol' as a first step. Given the above I think your ideas of running the daemon with no build dir are not 'minimal' and I'm convinced they're not viable. Someone would have to prototype the ideas, but I don't think that's a good use of time or energy right now (mostly because I think they're not viable). I recommend focussing on the tasks in my OP: http://thread.gmane.org/gmane.comp.lib.qt.creator/11794 >> Richer information about some semantics like 'task' and 'busy state' >> could also be provided in a similar way I expect, assuming those could be > defined. > > This is some basic functionality that we should get right soon, as this > can influence how long-running tasks need to be designed. Yes. Perhaps you can expand on what 'tasks' and 'busy state' you have in mind. Particularly if you can relate them to what is already in my github branch. > So "cmake --build" is out of scope for the daemon? I listed three ways cmake and an IDE could interface here: http://thread.gmane.org/gmane.comp.lib.qt.creator/11794/focus=15411 'triggering the build' would be a fourth. I don't think it needs to be part of the initial discussion of the design. Let's try to keep scope small so that we can get on common ground. > That is why I brought up progress information so early: They make things > complicated by there suddenly being several responses to a request and you > need a way to identify those. At that point simpler solutions tend to blow > up:-) Yes. BTW: I don't expect to get any part of the design of the protocol 'right' on the first iteration. I think we need to start everything that will need to be in the protocol, then iterate. Start high level ignoring details, and fill in the details as we iterate and can encode needs in unit tests. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] cmake daemon mode protocol
Hi Stephen,
Am 15.01.2016 00:38 schrieb "Stephen Kelly" :
> > * Start daemon-mode without any additional parameters
> >
> > * Daemon responds with "
> > {
> > "type": "handShake",
> > "version": "3.5.x",
> > "supportedProtocols": "3.5"
> > }
>
> As you discovered, the cmake binary might not even support the daemon mode
> at all.
That is really straight forward to detect: It will send malformed JSON on
stdout if daemon-mode is unsupported:-) The error message you get is not
JSON and definitely on of type handShake.
> I think what you are really asking for is 'the capabilities of the cmake
> binary'. In that sense, you might also want to know what generators are
> available, whether daemon mode is available (and with which protocol
> versions) etc.
>
> The issue at
>
> http://public.kitware.com/Bug/view.php?id=15462
>
> proposes to dump that information to stdout with a different -E command.
I do not really see the need for that.
Daemon-mode is easy to detect by the reply to the attempt to start it.
Everything else I expect to extract from within daemon mode.
> >
> > * Client Handshake:
> > {
> > "type": "handshake",
> > "requestProtocolVersion": "3.5"
> > }
> >
> > * Daemon:
> > {
> > "type:"handshake",
> > "protocolVersion": "3.5"
> > }
>
> Something like this is possible too. It's not far off what is already done
> in the branch.
I know, that is what this is based on after all. All I want is to be able
to start daemon-mode without having a builddir that is already associated
with a sourcedir.
> > At this point the normal operation should be possible.
> >
> > * Client:
> > {
> > "type": "setProjectInformation",
> > "sourceDirectory": "/some/path",
> > "cookie": "1"
> > }
> >
> > Being able to set the source directory is important so that a fresh
> > checkout can be configured. There is no build directory available at
> > that time...
>
> I understand the desire to inspect a cmake project without asking the user
> to specify a build dir,
You can safely read the CMakelists at this point, give syntax errors, maybe
even extract basic information like project name at this point.
> but cmake has to write build files somewhere in
> order to test the compiler etc.
If this works with a temporary build directory, then these files probably
do not need to litter the build directory in the first place:-)
> For this case, I suggest that if the user tries to 'open the source
> directory', you would use QTemporaryDir to build in a temporary location
and
> run the daemon there. I believe clion does something equivalent to that.
Hey, the idea is to improve on the current situation, not to perpetuate the
workarounds.
And how do I associate an empty directory as a builddir to a sourcedir?
How can I switch between builds when the daemon-mode is set up to work with
one builddir? Do I have to close it down and start a new daemon for the
next build?
That would be suboptimal, as all the built-in dependent state of the
project would need to get regenerated by the new daemon. Associating a
daemon with a sourcedir and being able to switch builddir would be nicer.
> Is that viable? I suppose you are suggesting that cmake do that instead of
> leaving it for clients to do?
It is doable. I just do not see why this should be necessary.
The daemon is not need to persist its state to disk all the time like
command-based cmake currently needs to.
> > * Daemon should do progress handling while reading the CMakeLists.txt
> > or whatever it does:
> > { "type": "progress", "state": "busy", "task": "someId",
> > "taskDescription": "Reading CMakeLists.txt files", "cookie": "1" }
> >
> > * Daemon sends occasional:
> > { "type": "progress", "task": "someId", progressTotal: 25,
> > progressCurrent: 12, "cookie": "1" }
>
> Yes. CMake can provide approximate values of how complete the configure
step
> is. See cmGlobalGenerator::AddMakefile. That could be exposed to the
daemon
> 'somehow', such as by defining some virtual callback interface.
>
> Richer information about some semantics like 'task' and 'busy state' could
> also be provided in a similar way I expect, assuming those could be
defined.
This is some basic functionality that we should get right soon, as this can
influence how long-running tasks need to be designed.
> > * Finally Daemon responds sends data as that is available now.
> > {
> > "type": "projectInformation",
> > "sourceDirectory": "/some/path",
> > "buildDirectory": "",
> > "projectName": "SomeProject",
> > "cookie": "1"
> > }
>
> At this point in your proposal, cmake has run the configure step in a
> temporary directory, right?
I am currently arguing with complete ignorance of how cmake works
internally:-) This is just what I think makes the most sense from an IDE
perspective.
> So it would be
>
> "buildDirectory": "/tmp/foo/bar"
>
> right?
No.
The daemon should be associated with a sourcedir, not a builddir. At th
Re: [cmake-developers] cmake daemon mode protocol
Tobias Hunger wrote:
> Hi Stephen,
>
> I have successfully build and run your cmake server mode changes and
> the python client script does work as advertised.
Thanks for doing that!
> I do have a couple of remarks about it. This is more intended as a
> starting point for discussion as a real proposal. Would something
> along these lines be possible:
> * Start daemon-mode without any additional parameters
>
> * Daemon responds with "
> {
> "type": "handShake",
> "version": "3.5.x",
> "supportedProtocols": "3.5"
> }
As you discovered, the cmake binary might not even support the daemon mode
at all.
I think what you are really asking for is 'the capabilities of the cmake
binary'. In that sense, you might also want to know what generators are
available, whether daemon mode is available (and with which protocol
versions) etc.
The issue at
http://public.kitware.com/Bug/view.php?id=15462
proposes to dump that information to stdout with a different -E command.
>
> * Client Handshake:
> {
> "type": "handshake",
> "requestProtocolVersion": "3.5"
> }
>
> * Daemon:
> {
> "type:"handshake",
> "protocolVersion": "3.5"
> }
Something like this is possible too. It's not far off what is already done
in the branch.
>
> At this point the normal operation should be possible.
>
> * Client:
> {
> "type": "setProjectInformation",
> "sourceDirectory": "/some/path",
> "cookie": "1"
> }
>
> Being able to set the source directory is important so that a fresh
> checkout can be configured. There is no build directory available at
> that time...
I understand the desire to inspect a cmake project without asking the user
to specify a build dir, but cmake has to write build files somewhere in
order to test the compiler etc.
For this case, I suggest that if the user tries to 'open the source
directory', you would use QTemporaryDir to build in a temporary location and
run the daemon there. I believe clion does something equivalent to that.
Is that viable? I suppose you are suggesting that cmake do that instead of
leaving it for clients to do?
> * Daemon should do progress handling while reading the CMakeLists.txt
> or whatever it does:
> { "type": "progress", "state": "busy", "task": "someId",
> "taskDescription": "Reading CMakeLists.txt files", "cookie": "1" }
>
> * Daemon sends occasional:
> { "type": "progress", "task": "someId", progressTotal: 25,
> progressCurrent: 12, "cookie": "1" }
Yes. CMake can provide approximate values of how complete the configure step
is. See cmGlobalGenerator::AddMakefile. That could be exposed to the daemon
'somehow', such as by defining some virtual callback interface.
Richer information about some semantics like 'task' and 'busy state' could
also be provided in a similar way I expect, assuming those could be defined.
> * Finally Daemon responds sends data as that is available now.
> {
> "type": "projectInformation",
> "sourceDirectory": "/some/path",
> "buildDirectory": "",
> "projectName": "SomeProject",
> "cookie": "1"
> }
At this point in your proposal, cmake has run the configure step in a
temporary directory, right? So it would be
"buildDirectory": "/tmp/foo/bar"
right?
> * Client:
> {
> "type": "setProjectInformation",
> "buildDirectory": "/some/other/path",
> "cookie": "2"
> }
As this sets a new build directory, cmake will have to run the configure
step from scratch in this new location.
> * Daemon does progress indication as above...
>
> * Daemon responds with project info:
> {
> "type": "progressInformation",
> "sourceDirectory": "/some/path",
> "buildDirectory": "/some/other/path",
> "projectName": "SomeProject",
> "cookie": "2"
> }
I must admit I'm not seduced by this idea as it seems to be that it should
be easy for an IDE to build in a temporary dir itself.
However, I think it makes sense to design the protocol to handle specifying
either a sourceDir or a buildDir (or both if from scratch). Starting daemon
mode without arguments as you suggest could work (though we would have to
see how that affects the protocol). Or with a different argument (eg
cmake -E daemon buildDir
cmake -E daemon_source sourceDir
) could work. I am also just brainstorming ideas, not proscribing anything.
> We might also want a "ping"/"pong" to detect if the daemon is still
> responding.
Yes, I had the same thought. I also considered self-terminating the daemon
if there is no message on the wire for N seconds, effectively requiring the
IDE to ping every N seconds if it is not doing anything else. I considered
this because otherwise the IDE might crash and leave zombie cmake daemons
running, but I didn't investigate that possibility.
> That would require the daemon to be able to handle
> multiple requests to be in flight... not sure that this is worth it.
Yes, adding a multi-threading requirement to the daemon might mak
[cmake-developers] cmake daemon mode protocol
Hi Stephen,
I have successfully build and run your cmake server mode changes and
the python client script does work as advertised.
I do have a couple of remarks about it. This is more intended as a
starting point for discussion as a real proposal. Would something
along these lines be possible:
* Start daemon-mode without any additional parameters
* Daemon responds with "
{
"type": "handShake",
"version": "3.5.x",
"supportedProtocols": "3.5"
}
* Client Handshake:
{
"type": "handshake",
"requestProtocolVersion": "3.5"
}
* Daemon:
{
"type:"handshake",
"protocolVersion": "3.5"
}
At this point the normal operation should be possible.
* Client:
{
"type": "setProjectInformation",
"sourceDirectory": "/some/path",
"cookie": "1"
}
Being able to set the source directory is important so that a fresh
checkout can be configured. There is no build directory available at
that time...
* Daemon should do progress handling while reading the CMakeLists.txt
or whatever it does:
{ "type": "progress", "state": "busy", "task": "someId",
"taskDescription": "Reading CMakeLists.txt files", "cookie": "1" }
* Daemon sends occasional:
{ "type": "progress", "task": "someId", progressTotal: 25,
progressCurrent: 12, "cookie": "1" }
* Daemon sends (probably optional...):
{ "type": "progress", "task": "someId", "state": "done", "cookie": "1" }
* Finally Daemon responds sends data as that is available now.
{
"type": "projectInformation",
"sourceDirectory": "/some/path",
"buildDirectory": "",
"projectName": "SomeProject",
"cookie": "1"
}
* Client:
{
"type": "setProjectInformation",
"buildDirectory": "/some/other/path",
"cookie": "2"
}
* Daemon does progress indication as above...
* Daemon responds with project info:
{
"type": "progressInformation",
"sourceDirectory": "/some/path",
"buildDirectory": "/some/other/path",
"projectName": "SomeProject",
"cookie": "2"
}
We might also want a "ping"/"pong" to detect if the daemon is still
responding. That would require the daemon to be able to handle
multiple requests to be in flight... not sure that this is worth it.
But if we can not do that, then we need to provide constant progress
information, which is also not great:-)
Not sure about the need for cookies (text set by the client and
repeated by the server in all its replies to that request). I do think
having them makes sense, even when only one request can be active at
any time: They make it so much easier to sanity-check communication.
Consistently having a "type" in all messages going back and forth is
probably a good idea for the same reason as having cookies.
A final, remark unrelated to the protocol: Ideally the daemon should
delay writing into (and even creating!) the build directory for as
long as possible (or till instructed to actually persist the
configuration). We do have many users that use Qt Creator to browse
foreign code and those tend to get upset when we write stuff to disk.
Best Regards,
Tobias
--
Powered by www.kitware.com
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Kitware offers various services to support the CMake community. For more
information on each offering, please visit:
CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On Tuesday, January 12, 2016 23:20:14 Milian Wolff wrote: > On Montag, 11. Januar 2016 23:22:23 CET Alexander Neundorf wrote: ... > > Stephens big approach will need some time until it is ready, while such a > > (relatively) simple thing can probably be done within one release cycle > > (but I don't have the time to do it). > > > > I don't know what the others here think about adding your json-approach or > > not. > > I don't think I'll have the required time to help here either, but I agree > with your reasoning: Your patch is the potentially quickest to get something > done. But it only fixes the usability issue of generators, it does not come > with a useable generator which we'll still need to write. I thought Aleix's patch was already quite good ? Alex -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
Brad King wrote: > The proposed daemon could help IDEs integrate with the CMake language > without re-implementing it, but we may not need such heavy infrastructure > for IDEs just to list targets and sources and allow users to edit them. Do you agree that that's an orthogonal use-case (add file to target) to what is being proposed (help me debug my cmake files)? If you agree that the declarative spec is orthogonal, then do you still have some other objection 'in principle' to the daemon approach? > I'm concerned that the memory usage of a daemon implementing the proposed > capabilities may be too large to be practical (at least without a major > redesign of certain structures that tend to duplicate substrings, or > some kind of out-of-core approach). If we reach a point where the memory consumption issue is the only blocker I can try to trap Milian in a room one of these days to help fix it. Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
Tobias Hunger wrote: > So could we please get a *documented* way that an IDE can *rely on to > be available* that provides basic information on a > project/configuration: > > * Source directory, build directory > * Files that belong to the project (sources, headers, custom cmake > modules, CMakeLists.txts, resources, ...) > * Files that belong to the build system (anything that needs watching > to re-generate the IDE-integration information at the right times) > * Targets that can be built > * Files that belong to each target > * Compiler flags > * Defines in effect > * Include paths used > * The compilers used to build those files Yes, that's the kind of thing my branch on github provides. That's why I worked on those things first. Did you look at my github branch already by the way? Thanks, Steve. -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On Monday, January 11, 2016 15:59:35 Aleix Pol wrote: ... > > Hi Stephen, everyone, > I've already discussed this in private with you. I think it's a good > idea and I'd like to make sure we can benefit from this. > > I'm unsure of the feasibility of the project though. you maybe remember that my main issue with your patch last year was that it's not a "generator", while it does what generators are for. After talking with Milian a bit several week ago, I understood that an important reason why the kdevelop team does not want a generator is that there is only one generator per build-tree, i.e. the user has to decide beforehand which IDE project files he wants. With the branch I created on github a few days ago, which I announced here too, I changed the way the extra generators are activated: they are not activated like the main generators anymore, but can be turned on and off using normal cmake cache options, so multiple generators (e.g. the generic json generator and let's say the Eclipse generator) can both be turned on, initially or later on. I think this should solve this problem. The work there (https://github.com/neundorf/CMake/tree/RefactorExtraGeneratorsExperimental ) is not ready for inclusing, but it shows how it can be done and can serve as a starting point/inspiration. Stephens big approach will need some time until it is ready, while such a (relatively) simple thing can probably be done within one release cycle (but I don't have the time to do it). I don't know what the others here think about adding your json-approach or not. Alex -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On 01/11/2016 09:59 AM, Aleix Pol wrote: > I'm unsure of the feasibility of the project though. Getting any > changes in cmake takes forever (this initiative started almost 2 years > ago already and it hasn't had any impact yet) This particular change will establish a much wider contract between CMake and IDEs than we have now, and it has never been clear what the best approach should be. That is why this change is taking so long. It was unfortunate that your (Aleix's) first iteration I reviewed and guided last year got so far and took you so much of your time before others came forward with criticisms of the approach, but I think in the long run we can do something better. > and we still need to proof the approach and make sure it has mechanisms > to provide retro-compatibility. At least to some extent. > > At the very least, I'd like to see cmake maintainers committing to the > initiative, to make sure I don't waste more of my time. Stephen's cmState refactoring has been a fantastic cleanup and improvement of the organization of CMake's implementation. It is a major step toward separating the configuration and generation steps and will be useful for many future directions of CMake. However, after needing to make fixes between 3.4.0 and 3.4.1 like these: cmState: Avoid accumulating policy stack storage for short-lived scopes https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=f21dc4a8 cmState: Avoid accumulating snapshot storage for short-lived scopes https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=5f860ebb I'm concerned that the memory usage of a daemon implementing the proposed capabilities may be too large to be practical (at least without a major redesign of certain structures that tend to duplicate substrings, or some kind of out-of-core approach). The proposed daemon could help IDEs integrate with the CMake language without re-implementing it, but we may not need such heavy infrastructure for IDEs just to list targets and sources and allow users to edit them. The current need for it is due to the fact that the build specification is in an imperative language instead of a declarative format. I think a better approach forward is that discussed in this thread: CMake alternative language http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/15339/focus=15374 Over there I propose moving the majority of the specification into a stateless format that IDEs could more easily load, edit, and save. While the daemon proposed here may have value, I think we should first resolve discussion of the approach proposed in the other thread. -Brad -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake daemon for user tools
On Sun, Jan 10, 2016 at 12:10 PM, Stephen Kelly wrote: > > Hello, > > I've been working on adding a daemon mode for cmake to provide > information to user tools - such as IDEs - about the buildsystem. > > Following the discussion about providing metadata for IDEs to consume > I proposed creating a long-running process which would provide a protocol > to access information about the buildsystem, and about the content of the > cmake files themselves: > > > http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004 > > The original post in that thread lists design goals. > > This design is independent of any generator, and compared to the solution of > writing json to a file, this solution doesn't require generating > anything that is > not asked for. This also side-steps the problem of 'stale' files remaining > in the build directory, and the problem of needing to trigger re-generation > of the file, or watching it for changes. This solution also allows > parameters (eg the config) to be specified in requests, side-stepping a > difficulty we encountered before to represent things like that. This > solution also does not require actually generating the buildsystem > files. Only the configure and compute steps of cmake are executed. > > I am scheduled to give a talk at FOSDEM about the feature and how user > tools can interact with it: > > https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code > > with a preview in Berlin: > > http://www.meetup.com/berlincplusplus/events/227896427/ > > I have now also pushed a branch to my github clone with the start of the > server mode: > > https://github.com/steveire/CMake/commits/cmake-daemon > > Currently the branch only makes targets and backtraces etc available via > the protocol. I have also created a plan for extending the protocol to > make code completion and variable debugging possible, and analysed the > depedencies of the tasks: > > http://www.steveire.com/cmake-daemon-tasks.png > > However, I can't complete those tasks myself: I don't have > relevant experience building IDEs to know how best to design the > protocol, what IDE tools really need, how to design a fail-safe parser > etc. Additionally, I think a design which incorporates design ideas > and implementation from more than one person will be better in the end. > > So, this needs to be a collaborative effort if it is to go anywhere, with > more people writing commits in the cmake repo. > > The above (copied below) task list would be enough to get a read-only > browser of a cmake project quite quickly. > > Following that, an effort would be needed to handle the difference > between the dirty state of an editor and the on-disk file content. That's > a solvable problem and together with filesystem notifications provided > by libuv, it will lead to features like code completion. > > Also necessary eventually would be to make the cmake parser more > fail-safe, so that information is still available if the user > had not yet closed a command argument list for example, and make the > processing more fail-safe in the case where they have written > add_subdirectory(foo), but have not yet created foo/CMakeLists.txt. > > I would be looking to people who work on IDEs for guidance on how to do > things like that, as some of them are likely already for handling C++ > code in the IDEs. > > So, this needs to be a collaborative effort. I can provide guidance to > anyone who wishes to participate in changing the source code of CMake > to make it possible. > > Is anyone interested enough in this potential cmake feature to > join the development effort? > > Thanks, > > Steve. > > > * Provide basic daemon mode with event loop and communication protocol > -- Written, but not reviewed > * Provide protocol to query the buildsystem structure > -- Written, but not complete or reviewed > * Extend cmListFileParser to record location of start and end of commands > * Rewrite cmIfCommand to record which branches are executed > * Store the location of start and end of user-defined functions > * Provide protocol to query for unexecuted lines > * Define a new snapshot type to occur at an arbitrary point > * Create snapshots after branching to other files (after an include() or > find_package() or add_subdirectory etc) > * Create snapshots after points with external effects (file(WRITE), > configure_file(), execute_process etc) > * Find most recent snapshot occuring before a specified line > * Implement feature to start at a specific snapshot and execute commands > up to a particular line > * Make server find correct snapshot to start at and execute required > commands up to specified line > * Provide protocol to dump variable defintions at a particular line > * Extract implementation of IssueMessage from cmMakefile and > cmLocalGenerator into cmake > * Extract computed target properties from cmTarget::GetProperty into > cmComputedTargetProperties > * Implement cmGeneratorTarget::GetPro
[cmake-developers] CMake daemon for user tools
Hello,
I've been working on adding a daemon mode for cmake to provide
information to user tools - such as IDEs - about the buildsystem.
Following the discussion about providing metadata for IDEs to consume
I proposed creating a long-running process which would provide a protocol
to access information about the buildsystem, and about the content of the
cmake files themselves:
http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=13004
The original post in that thread lists design goals.
This design is independent of any generator, and compared to the solution of
writing json to a file, this solution doesn't require generating
anything that is
not asked for. This also side-steps the problem of 'stale' files remaining
in the build directory, and the problem of needing to trigger re-generation
of the file, or watching it for changes. This solution also allows
parameters (eg the config) to be specified in requests, side-stepping a
difficulty we encountered before to represent things like that. This
solution also does not require actually generating the buildsystem
files. Only the configure and compute steps of cmake are executed.
I am scheduled to give a talk at FOSDEM about the feature and how user
tools can interact with it:
https://fosdem.org/2016/schedule/event/enabling_gui_tools_for_cmake_code
with a preview in Berlin:
http://www.meetup.com/berlincplusplus/events/227896427/
I have now also pushed a branch to my github clone with the start of the
server mode:
https://github.com/steveire/CMake/commits/cmake-daemon
Currently the branch only makes targets and backtraces etc available via
the protocol. I have also created a plan for extending the protocol to
make code completion and variable debugging possible, and analysed the
depedencies of the tasks:
http://www.steveire.com/cmake-daemon-tasks.png
However, I can't complete those tasks myself: I don't have
relevant experience building IDEs to know how best to design the
protocol, what IDE tools really need, how to design a fail-safe parser
etc. Additionally, I think a design which incorporates design ideas
and implementation from more than one person will be better in the end.
So, this needs to be a collaborative effort if it is to go anywhere, with
more people writing commits in the cmake repo.
The above (copied below) task list would be enough to get a read-only
browser of a cmake project quite quickly.
Following that, an effort would be needed to handle the difference
between the dirty state of an editor and the on-disk file content. That's
a solvable problem and together with filesystem notifications provided
by libuv, it will lead to features like code completion.
Also necessary eventually would be to make the cmake parser more
fail-safe, so that information is still available if the user
had not yet closed a command argument list for example, and make the
processing more fail-safe in the case where they have written
add_subdirectory(foo), but have not yet created foo/CMakeLists.txt.
I would be looking to people who work on IDEs for guidance on how to do
things like that, as some of them are likely already for handling C++
code in the IDEs.
So, this needs to be a collaborative effort. I can provide guidance to
anyone who wishes to participate in changing the source code of CMake
to make it possible.
Is anyone interested enough in this potential cmake feature to
join the development effort?
Thanks,
Steve.
* Provide basic daemon mode with event loop and communication protocol
-- Written, but not reviewed
* Provide protocol to query the buildsystem structure
-- Written, but not complete or reviewed
* Extend cmListFileParser to record location of start and end of commands
* Rewrite cmIfCommand to record which branches are executed
* Store the location of start and end of user-defined functions
* Provide protocol to query for unexecuted lines
* Define a new snapshot type to occur at an arbitrary point
* Create snapshots after branching to other files (after an include() or
find_package() or add_subdirectory etc)
* Create snapshots after points with external effects (file(WRITE),
configure_file(), execute_process etc)
* Find most recent snapshot occuring before a specified line
* Implement feature to start at a specific snapshot and execute commands
up to a particular line
* Make server find correct snapshot to start at and execute required
commands up to specified line
* Provide protocol to dump variable defintions at a particular line
* Extract implementation of IssueMessage from cmMakefile and
cmLocalGenerator into cmake
* Extract computed target properties from cmTarget::GetProperty into
cmComputedTargetProperties
* Implement cmGeneratorTarget::GetProperties in terms of
cmComputedTargetProperties
* Move implementation of cmTarget::{Get,Set,Append}Property to cmState
* Extract all IssueMessage parts of cmake into a cmMessenger
* Extract the cmSystemTools::Message part of cmMessenger into a virtual
m
