Hello CMake Developers, Stephen Kelly and me met last Tuesday to talk about the daemon-mode patch we both have been working on. It was a very productive meeting: We managed to resolve almost all the differences of opinion we had.
Here is a short summary of what we talked about: 0) General * We both want a daemon mode:-) * Some names could be improved. Suggestions welcome. * Having cmMessanger would be nice and will make things easier. * We agreed that we should get the basic infrastructure part reviewed first, followed by the functionality up to the project structure (codemodel). * We both think it only makes sense to merge the infrastructure part into cmake (if it passes review first of course) once we have some functionality that is genuinely useful. So we want to aim at having the infrastructure and the codemodel merged in one go. 1) Infrastructure: 1.1 Communication: * Clients should talk to daemon mode via a socket (libuv supports that), but we also want to support stdin/stdout at least for a "debug mode" * There should be a separate command line switch to start stdin/out mode and for socket mode * This is needed before it makes sense to merge the patch set. 1.2 Protocol versions * We need the daemon-mode to support different protocol versions. These need to be versioned independently of the cmake binary itself and we will try to keep the protocol versions working as long as that is reasonable, but would try to support them for at least a year or two (if there are no overriding concerns). * We want to have the option to mark protocol versions as experimental. Experimental protocol versions come with *NO* guarantees and can change *AT ANY TIME*. If a experimental protocol ends up in a cmake release there still are no guarantees. I would like to see experimental protocol versions in cmake releases, since that makes it really easy for other parties to evaluate changes. In my experience hardly anybody builds custom branches:-/ All this would need to be communicated by cmake e.g. in the release notes. 1.3 Basic Protocol * Client/daemon talk via JSON messages. * All messages have a "type" with a string specifying the type of message. * We (mostly) have a request/reply model. The client sends a request message and will receive a reply or an error. * Requests contain a type, and an optional cookie (which will be returned together with any message sent by the daemon in response to the request). A cookie can be any JSON object. * A reply is a JSON message of type "reply". It contains the cookie of its request and any data that the daemon wants to send its client. * An error is a JSON message of type "error". It contains the cookie of its request and an "errorMessage". * There are more JSON messages for progress information and messages, etc. and we can add more as needed. TO DECIDE: * Currently a reply (and error and all other messages in response to a request) contain a "inReplyTo" with the type string from the request triggering the reply. Stephen thinks that is not necessary and argues anything that is not strictly necessary should not be in the first version of the protocol. I like having this as it makes it helps to identify what a reply/error is related to. I often paste a whole string of requests into the daemon and if there is an error it is really easy to see how for things got. * Should we have a split between header (cookie/type/inReplyTo) and data? Stephen suggested passing all data that is from/for the client in a "body" attribute to make messages more uniform. I think that is just adding one more level into the JSON file for no real benefit. 2) Basic Command Set 2.1) Reset (Reset to a state right after daemon start) * Remove this. This is error prone as data structures may not be cleaned out properly. 2.2) globalSettings (Query generators, debug flags, etc.) * Move generators and other static information into a separate cmake commandline invocation (http://public.kitware.com/Bug/view.php?id=15462): * Generators with information on them * CMake version information * Module path * Daemon mode support all should be available in a machine readable way from that command. TODO: Figure out how to get all the generator information, e.g. which values the flags some generator taken can have (architecture, etc.). Stephen volunteered to check into this. * We can report the selected generator/extragenerator here. * Undecided about whether it makes sense to report source/builddirectory here 2.3) setGlobalSettings (set the information from globalSettings) * This is currently used to set sourcedirectory, builddirectory and generator. These three should be passed in via the command line instead, again to avoid trouble with users switching to different build directories/etc. while the daemon is running, which might trigger problems with state cleanup, etc. * The rest (trigger developer warnings, etc.) seems fine. * Generator/build- and sourcedirectory may not be set (even if reported via globalSettings). Extragenerator can be set. 2.4) configure (Trigger cmake to configure a builddirectory) * Fine, handles "-DSOMETHING=other", etc. via normal cmake commandline parsing. TODO: Validate that it handles broken input with decent error reporting. 2.5) generate * Rename this to "compute", which is what this actually triggers in cmake. 2.6) project (Return project structure) * add backtrace information to targets * Rename to "codemodel" to make the use-case for the data more clear. * Focus on providing data for the use-case of feeding a code model. * Make the configuration the top-level, then report projects, targets and files (in groups) beneath that. Currently projects are the top level, then there is configurations, targets and files. Switching this around should offer more flexibility to change cmake internals going forward. * We do not want cmake internals to leak into the output. So do not report cmake features, etc. * We try to minimize the assumptions of cmake internals in the implemention to allow cmake room to innovate without breaking changes to this. * Policies should not be visible at this level and do not need to be exposed to clients. FUTURE WORK: Stephen and I did not agree on how to handle other but similar use-cases that need similar information as to what is in this view and we will need to revisit this topic as new use-cases will get added in the future: Stephen prefers having special commands for each use-case, returning output tailored to that use-case. I would prefer having one command to gather the complete state of the project structure and then allow the user to limit what is returned from this command (e.g. only show one configuration/target/types of targets/etc. and options to filter out e.g. backtraces from the data that is returned). Stephens approach has the advantage that it is easier to use and more predictable what is returned. My approach allows client authors more flexibility as to which information they need without having to ask cmake developers to add a new command/change an existing one. But this is for the future: For the first iteration we want to concentrate on data relevant to feeding a code model. TODO: * Figure out whether a PDB file is actually going to be build. Currently the PDB file is added to the list of artifacts whenever there is a .lib file, which is probably wrong. Brad: Do you have an idea how to get this information? 2.7 cache (report contents of CMakeCache.txt file) * Review by other potential users would be appreciated, but no obvious problems seen. 2.8 buildsystem (return cmake files) * Return a list of cmake files (CMakeLists.txt, etc.) in different categories (in source directory/in build directory/in the system). * Stephen suggested to rename this to "cmake_rerun_depends". I am not sure that name is much better than what is currently used. 3.0 Features from Stephen's branch that need porting We only skimmed this, as many of his highlights need refactoring in cmake to land first. 3.1 parse * This duplicates the parser and will need some refactoring. Status Information: I started to add some tasks to github to track what needs to be done before proposing the daemon-mode branch for release. This can be viewed here: https://github.com/hunger/CMake/issues Best Regards, Tobias -- Tobias Hunger, Senior Software Engineer | The Qt Company The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB 144331 B -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
