Re: [cmake-developers] How to handle configurations (Was: Generating buildsystem metadata from CMake)
Tobias Hunger wrote: Hi Stephen, On Sat, Mar 21, 2015 at 10:56 AM, Stephen Kelly steve...@gmail.com wrote: So, the design question I have is: * Is lots of repetition ok in the metadata file? If it can not be avoided, then so be it. It can be avoided at the cost of computation. I thought that was more clear in my mail. The answer will be re-used in many other places. For example, if I have a CMakeLists like add_executable(main file1.cpp ... fileM.cpp ) target_include_directories(main ...) target_compile_definitions(main ...) then we could either write the definitions and includes once (associated with the target), or we could repeat them all M times (once for each file). That could be lots and lots of repetition, repeated N times, once for each configuration. ... or define a group of files, put all the files into that group and add the other relevant flags. If there is a file that needs something different, then just add that as a separate group. This still just pushes a computation requirement to you the consumer of the file. The reason I'm asking about duplication versus computation is because of what you wrote: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=12750 Tobias Hunger wrote: Can't we go for a syntax where the files and everything that applies to them are grouped so that there is no need to reimplement the logic found in cmake (with added bugs:-)? If the concern that computation means bugs is the primary concern then we should duplicate a lot of information. Also, the outcome of how we should generate compile flags could have an influence on what we duplicate. Anton will hopefully respond. I don't believe we'll have a huge problem with bugs processing the json. It will have a schema and documentation telling you how to consume it. If it is consumed in multiple steps (eg with processing of your 'source group' idea etc), then I think that's fine. We simply document how it's done. It's no different to any other data format in that sense. A spec is needed to implement the logic of parsing json too, and there are many json parsers out there. I'm sure some have hard-to-reach bugs, but all software has hard-to- reach bugs. In the normal case where all files have the same flags applied the overhead should be close to the minimum. And it still allows for having different settings for different groups of files without the IDE needing to combine settings. E.g. if main.cpp defined TEST=1, while the main-target defined TEST=2, which one will win? I guess this relates to why Anton wants the actual command line. That's undefined behavior. I guess it could be made defined. I filed http://public.kitware.com/Bug/view.php?id=15472 Please do not require all IDEs to implement logic found in cmake. There will always be corner cases where this will fail, leading to a broken code model in the IDE. I agree we don't want to require implementing logic found in CMake. I do think we should be allowed to require 'implementing well defined and specified logic', as you do too judging by your source groups idea. Attached you find something that is close to what I want. I moved the sections in your version of that file around a bit in a text editor, so this might not be valid json:-). Thanks for this. Actually I guess the generated file for that test is too big to do sensible comparisons. I basically want to see a proposal which is more complete than many different ideas of snippets and lots of '...', and I want to see how a consumer would process the file. Maybe we should start with a simpler CMake file. This does not really cover the conditions though, as those were not in your file either. The conditions are handled in my file. Files which do not match the condition all go into the 'excluded files' group. I implemented it that way because of this quote from you: http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11323 I wonder how those kinds of conditions would need to be represented in the ProjectTargets.json file. This is actually a bit too detailed for my needs:-) I want to know which files are part of the project and which are part of the current build. At least Qt Creator does not need information on which conditions to be met for a file to become part of the current build. The 'object sources' are files which will be compiled. The 'excluded' sources will not be compiled, but they are 'part of the project' as you seemed to describe above before. Also, note that in my json file, there is not a generic 'files' property because cmake has more information than just 'files' - it has information about which compiler is used for the files to compile objects, which files are not going to be compiled at all etc. My design gives you that information, but a 'files' array discards it. i do not see configurations as much of a problem in
Re: [cmake-developers] How to handle configurations (Was: Generating buildsystem metadata from CMake)
Hi Stephen, On Sat, Mar 21, 2015 at 10:56 AM, Stephen Kelly steve...@gmail.com wrote: So, the design question I have is: * Is lots of repetition ok in the metadata file? If it can not be avoided, then so be it. The answer will be re-used in many other places. For example, if I have a CMakeLists like add_executable(main file1.cpp ... fileM.cpp ) target_include_directories(main ...) target_compile_definitions(main ...) then we could either write the definitions and includes once (associated with the target), or we could repeat them all M times (once for each file). That could be lots and lots of repetition, repeated N times, once for each configuration. ... or define a group of files, put all the files into that group and add the other relevant flags. If there is a file that needs something different, then just add that as a separate group. In the normal case where all files have the same flags applied the overhead should be close to the minimum. And it still allows for having different settings for different groups of files without the IDE needing to combine settings. E.g. if main.cpp defined TEST=1, while the main-target defined TEST=2, which one will win? Please do not require all IDEs to implement logic found in cmake. There will always be corner cases where this will fail, leading to a broken code model in the IDE. I personally prefer minimising the repetition, as I did in the unit test I pushed to my clone. I have the feeling you guys have not looked at the unit test I pushed to my clone. Please go ahead and do that if not. Also, when proposing json formats, please imagine how your proposal would represent the data in Tests/Metadata/CMakeLists.txt. I am not sure I understand cmake well enough to do that correctly:-/ But I'll give it a try. I'd actually prefer having a configurations key with the list of the configurations an object applies to. That can be optional if the object is relevant to all configurations (default in single-configuration generators;-). This key would then need to be applicable to targets as well as the group of source files. So that would be something like this: Could you post what would be generated by your proposed structure for the unit test file in my branch? That would most likely avoid quite a bit of duplication in the file(s). How does the design goal of 'avoid duplication' weigh against the design goal of 'don't make me combine multiple properties together to get relevant information (eg target defines and additional source defines)', in general? We can either go for one extreme or the other, but I don't think it makes sense to go for something in between. That would be the worst of both worlds. So, for the proposals which are on the table, please either use the Metadata/CMakeLists.txt file in my clone, or post an alternative together with the json which you propose we should generate for it, and a prototype or pseudo-implementation of a parser for that json. Attached you find something that is close to what I want. I moved the sections in your version of that file around a bit in a text editor, so this might not be valid json:-) This does not really cover the conditions though, as those were not in your file either. Note how the main.cpp is handled differently from your proposal. i do not see configurations as much of a problem in such a setup: Just add them as tags to each group of files (and each target). That way the configurations can be handled just as the other differentiators (language, type) in the file groups/targets. That should reduce the duplication, still be pretty readable and should be reasonably simple to parse. Best Regards, Tobias cmake-metadata-Linux-GNU-4.9.json Description: application/json -- Powered by www.kitware.com Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Kitware offers various services to support the CMake community. For more information on each offering, please visit: CMake Support: http://cmake.org/cmake/help/support.html CMake Consulting: http://cmake.org/cmake/help/consulting.html CMake Training Courses: http://cmake.org/cmake/help/training.html Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/cmake-developers
[cmake-developers] How to handle configurations (Was: Generating buildsystem metadata from CMake)
This seems to be to be a design point which we are blocked on. There are
many different ideas floating around.
I think we need to talk about what would be generated for actual CMakeLists
files, and how tools would parse the result. I suggest starting with the
Tests/Metadata/CMakeLists.txt file in my clone.
I think we need to see both json files and prototype parsers in some
mainstream language or framework (Qt5,Python etc) in order to see whether a
particular proposal is viable.
For my proposal there is already a generated json file checked in as an
example, and a python class for extracting the information.
The requirements for the metadata file include
1) Make it easy to get relevant information without extra computation/logic.
2) Be as DRY as possible.
3) Design something that is possible for cmake to generate.
The first two requirements are obviously in direct conflict.
The third will I'm sure be a limitation compared to what you desire. If some
part of what you propose is impossible, we'll either have to design a way to
make it possible, or accept that it is impossible and update the proposal.
Hopefully this will converge on a design which provides both what is needed
and what is possible.
Implementing it will be easy then.
Tobias Hunger wrote:
Anton Makeev wrote:
As a side note, it seems more natural to me to have one json file with
one or several configurations listed, providing that there is also shared
project info that should be in that files.
something like that:
project: ProjectName
configurations: {
{
name: Debug
targets: {...}
},
{
name: Release
targets: {...}
}
...
}
If we do this, then we have N repetitions of exactly the same data, or
almost exactly the same data. It could be a lot of data. Is that a problem?
For machine reading it is probably not a problem. All the repetition does
make it harder for human reading though of course. I also don't know at what
point does it become slower for the IDE to have to parse ~4-6N times the
amount of data. If the configuration is a 'mode' and only one configuration
is active at a time, you're not using the other stuff anyway.
If you want all the data for all configurations (even inactive ones) for
some reason, then the repetition probably doesn't matter at all. Even with
my proposal of separate files, you would want to load all the files anyway.
Is that the case?
So, the design question I have is:
* Is lots of repetition ok in the metadata file?
The answer will be re-used in many other places. For example, if I have a
CMakeLists like
add_executable(main
file1.cpp
...
fileM.cpp
)
target_include_directories(main ...)
target_compile_definitions(main ...)
then we could either write the definitions and includes once (associated
with the target), or we could repeat them all M times (once for each file).
That could be lots and lots of repetition, repeated N times, once for each
configuration.
I personally prefer minimising the repetition, as I did in the unit test I
pushed to my clone. I have the feeling you guys have not looked at the unit
test I pushed to my clone. Please go ahead and do that if not. Also, when
proposing json formats, please imagine how your proposal would represent the
data in Tests/Metadata/CMakeLists.txt.
Anton, in your proposal, given that realistically Makefile and Ninja
generators are not going to become aware of multiple configurations before
this feature reaches master, what should they generate?
project: ProjectName
configurations: {
{
name: WHAT_WAS_IN_CMAKE_BUILD_TYPE
targets: {...}
}
or
project: ProjectName
targets: {...}
or
project: ProjectName
configuration: WHAT_WAS_IN_CMAKE_BUILD_TYPE
targets: {...}
?
I'd actually prefer having a configurations key with the list of the
configurations an object applies to.
That can be optional if the object is relevant to all configurations
(default in single-configuration generators;-).
This key would then need to be applicable to targets as well as the
group of source files.
So that would be something like this:
Could you post what would be generated by your proposed structure for the
unit test file in my branch?
That would most likely avoid quite a bit of duplication in the file(s).
How does the design goal of 'avoid duplication' weigh against the design
goal of 'don't make me combine multiple properties together to get relevant
information (eg target defines and additional source defines)', in general?
We can either go for one extreme or the other, but I don't think it makes
sense to go for something in between. That would be the worst of both
worlds.
So, for the proposals which are on the table, please either use the
Metadata/CMakeLists.txt file in my clone, or post an alternative together
with the json which you propose we should generate for it, and a prototype
or pseudo-implementation of a parser for that json.
Thanks,
