subject:"\[cmake\-developers\] Generating buildsystem metadata from CMake"

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-10-13 Thread Stephen Kelly

Alexander Neundorf wrote:

> Maybe this is of interest: the Eclipse CDT developers are currently
> working on improved support for cmake:
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=350206
> http://dev.eclipse.org/mhonarc/lists/cdt-dev/msg29621.html

Yes, I noticed that too a few days ago.

My work should make reading the CMakeLists.txt file not necessary, as they 
seem to intend to do.

I'll ping them when I have something to show.

Thanks,

Steve.

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-10-12 Thread Alexander Neundorf

Hi,

On Saturday, July 25, 2015 20:33:46 Stephen Kelly wrote:
> Aleix Pol wrote:
> > On Sat, Apr 18, 2015 at 11:56 AM, Stephen Kelly
> > 
> >  wrote:
> >> Stephen Kelly wrote:
> >>> The aim is to generate a structured file containing metadata 
relating
> >>> the buildsystem.
> >> 
> >> I've been quiet on this thread for a while, so I think it is time for an
> >> update.
> >> 
> >> I became more ambitious in mid March and started prototyping a
> >> more-complete design for CMake IDE integration.
> > 
> > Hi Stephen,
> > Is there any news on the subject?
> 
> I have been working on cleaning up cmake
> 
>  $ git log --oneline --author=steveire --since="April 1" | wc -l
>  472
> 
> I've made lots of progress toward separating the configure and 
generation
> steps (required prerequisite for server features), but no working 
prototype
> ready to show yet I'm afraid.

Maybe this is of interest: the Eclipse CDT developers are currently 
working on improved support for cmake:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=350206
http://dev.eclipse.org/mhonarc/lists/cdt-dev/msg29621.html

But if I understand them correctly, this is not about improving support for 
projects which use cmake without eclipse, but to use cmake as the tool 
to generate the makefiles in Eclipse projects. But maybe I got that 
wrong.

Alex

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-07-25 Thread Aleix Pol

On Sat, Apr 18, 2015 at 11:56 AM, Stephen Kelly steve...@gmail.com wrote:
 Stephen Kelly wrote:

 The aim is to generate a structured file containing metadata relating the
 buildsystem.


 I've been quiet on this thread for a while, so I think it is time for an
 update.

 I became more ambitious in mid March and started prototyping a more-complete
 design for CMake IDE integration.

 Instead of teaching CMake to generate a metadata file, I want to add a
 server mode to CMake, so that IDEs can run the server in the build directory
 and query it for buildsystem metadata, code completion, etc. It can be
 implemented efficiently and it doesn't have to re-configure the build from
 the beginning each time. A lot of things become possible with such a design,
 but I'm being economic with details because I don't want to make promises I
 don't deliver in the end, even though I have high confidence that I actually
 can :).

 The good news is that I have done some proof of concept work on all aspects
 of this, but I don't yet have one branch which contains all of the
 individual proof of concepts combined. Doing this properly requires
 refactoring CMake quite a bit, which I've already been doing for a while to
 create a cmState class.

 The bad news is that because this is more ambitious, it will take more time
 and will not be part of the CMake 3.3 release early this summer. I do
 believe it can be implemented for the following release 3 months later
 though.

 Here's some prior art in other tools:

  http://thread.gmane.org/gmane.comp.compilers.clang.devel/21780
  https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html
  http://thread.gmane.org/gmane.comp.lib.boost.build/26905

 We can use JSON as the wire protocol, and we can talk about exactly what the
 protocol will be once the refactoring of CMake is done (that will take some
 more weeks). I'll post a more complete design proposal and my prototype at
 that point too.

 Thanks,

 Steve.


 --

 Powered by www.kitware.com

 Please keep messages on-topic and check the CMake FAQ at: 
 http://www.cmake.org/Wiki/CMake_FAQ

 Kitware offers various services to support the CMake community. For more 
 information on each offering, please visit:

 CMake Support: http://cmake.org/cmake/help/support.html
 CMake Consulting: http://cmake.org/cmake/help/consulting.html
 CMake Training Courses: http://cmake.org/cmake/help/training.html

 Visit other Kitware open-source projects at 
 http://www.kitware.com/opensource/opensource.html

 Follow this link to subscribe/unsubscribe:
 http://public.kitware.com/mailman/listinfo/cmake-developers

Hi Stephen,
Is there any news on the subject?

Aleix
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-07-25 Thread Stephen Kelly

Aleix Pol wrote:

 On Sat, Apr 18, 2015 at 11:56 AM, Stephen Kelly
 steve...@gmail.com wrote:
 Stephen Kelly wrote:

 The aim is to generate a structured file containing metadata relating
 the buildsystem.


 I've been quiet on this thread for a while, so I think it is time for an
 update.

 I became more ambitious in mid March and started prototyping a
 more-complete design for CMake IDE integration.

 Hi Stephen,
 Is there any news on the subject?

I have been working on cleaning up cmake 

 $ git log --oneline --author=steveire --since=April 1 | wc -l
 472

I've made lots of progress toward separating the configure and generation 
steps (required prerequisite for server features), but no working prototype 
ready to show yet I'm afraid.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-18 Thread Stephen Kelly

Stephen Kelly wrote:

 The aim is to generate a structured file containing metadata relating the
 buildsystem.


I've been quiet on this thread for a while, so I think it is time for an 
update.

I became more ambitious in mid March and started prototyping a more-complete 
design for CMake IDE integration. 

Instead of teaching CMake to generate a metadata file, I want to add a 
server mode to CMake, so that IDEs can run the server in the build directory 
and query it for buildsystem metadata, code completion, etc. It can be 
implemented efficiently and it doesn't have to re-configure the build from 
the beginning each time. A lot of things become possible with such a design, 
but I'm being economic with details because I don't want to make promises I 
don't deliver in the end, even though I have high confidence that I actually 
can :).

The good news is that I have done some proof of concept work on all aspects 
of this, but I don't yet have one branch which contains all of the 
individual proof of concepts combined. Doing this properly requires 
refactoring CMake quite a bit, which I've already been doing for a while to 
create a cmState class.

The bad news is that because this is more ambitious, it will take more time 
and will not be part of the CMake 3.3 release early this summer. I do 
believe it can be implemented for the following release 3 months later 
though.

Here's some prior art in other tools:

 http://thread.gmane.org/gmane.comp.compilers.clang.devel/21780
 https://sourceware.org/gdb/onlinedocs/gdb/GDB_002fMI.html
 http://thread.gmane.org/gmane.comp.lib.boost.build/26905

We can use JSON as the wire protocol, and we can talk about exactly what the 
protocol will be once the refactoring of CMake is done (that will take some 
more weeks). I'll post a more complete design proposal and my prototype at 
that point too.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-06 Thread Ben Boeckel

On Mon, Apr 06, 2015 at 19:08:37 +0200, Stephen Kelly wrote:
 AFAICT, CMake does still know the command line in your output-target-flags-
 once-in-ninja branch. 
 
 You just need to compute two outputs - once for writing the Ninja file 
 (containing variables), and again for writing the content for the metadata 
 (not containing variables), right?

Yes; I forget how much time was saved by not dealing with them at all.
Though if that path is only taken if this file is needed, it seems like
all the time required for this file may be more than the time saved
there anyways.

Thanks,

--Ben
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-06 Thread Ben Boeckel

On Sat, Apr 04, 2015 at 14:32:51 +0200, Stephen Kelly wrote:
 On 04/02/2015 10:29 AM, Anton Makeev wrote:
  Just to make sure, you mentioned that this file should be deprecated with 
  this new metadata.json,
  is that correct?
 
 I'm not sure I mentioned that, just that it's an externally defined
 format and we didn't want to try to extend it, but instead define a
 different format. The new format will hopefully make it unnecessary for
 IDEs to parse the compile-commands.json. However, there are other users
 of the compile-commands.json, so it will remain and the feature for
 generation of it will not be deprecated.

Hmm. I have a branch which refactors some things in the build.ninja file
into Ninja-level variables which then means that CMake itself doesn't
know the command line anymore. I was using `ninja -t compdb` to provide
the compile_commands.json file. This will conflict with writing out this
file :/ .

That branch has some nice performance improvements (since the Ninja file
reduced its size considerably), but maybe I could plumb it around this
and have it only available without this extra build info?

And just to be sure (since I have been only loosely following the thread
in chunks), writing this file will be completely optional right?

--Ben
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-06 Thread Stephen Kelly

Ben Boeckel wrote:

 On Sat, Apr 04, 2015 at 14:32:51 +0200, Stephen Kelly wrote:
 However, there are other users
 of the compile-commands.json, so it will remain and the feature for
 generation of it will not be deprecated.

Or, to be more clear - it doesn't need to be deprecated as a result of this 
discussion or feature.
 
 Hmm. I have a branch which refactors some things in the build.ninja file
 into Ninja-level variables which then means that CMake itself doesn't
 know the command line anymore.

AFAICT, CMake does still know the command line in your output-target-flags-
once-in-ninja branch. 

You just need to compute two outputs - once for writing the Ninja file 
(containing variables), and again for writing the content for the metadata 
(not containing variables), right?

 And just to be sure (since I have been only loosely following the thread
 in chunks), writing this file will be completely optional right?

Yes, it will be possible to run cmake without generating metadata.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-04 Thread Stephen Kelly

On 04/02/2015 10:29 AM, Anton Makeev wrote:
 I see, but how will having include_directories help ‘Add include 
 directory’-like actions?
 I mean, simply having this list (without additional info like backtraces) 
 doesn’t seem to help much.

We'll have to see :).

 So, in the end, we probably need tooling to be able to get the information 
 in both forms. Ie, as an array of directories, and as a string of compiler 
 command line options.

 We already have the latter in the form of compile-commands.json. One of the 
 reasons I wonder if you are using that already is to know whether 

 1) That is already a good enough solution 
 or 
 2) The new metadata file should contain similar information

 I'm wondering if we can learn anything from your experience with the 
 existing file.
 I’m afraid we don’t have much to contribute here, since we don’t use this 
 file at the moment.
 After a brief investigation, it seems to have the necessary command lines we 
 expect in the form
 that can be easily parsed (apart from  -o path -c path that will need to 
 be cut out, but it’s not a big deal).

I'm sure we can generate content without those lines if we refactor the
Module files a bit.


 Just to make sure, you mentioned that this file should be deprecated with 
 this new metadata.json,
 is that correct?

I'm not sure I mentioned that, just that it's an externally defined
format and we didn't want to try to extend it, but instead define a
different format. The new format will hopefully make it unnecessary for
IDEs to parse the compile-commands.json. However, there are other users
of the compile-commands.json, so it will remain and the feature for
generation of it will not be deprecated.


 Does that make sense? That means you would read the all_compile_flags for 
 the specific source file if it is present, and use the all_compile_flags 
 for the target otherwise (which you already have cached).
 That sounds like a brilliant idea. This way we’ll have minimum duplication 
 and minimum logic required on
 the IDE-side.


Great. Good to know this will work for you. I'll integrate the idea into
the next design iteration.

Thanks,

Steve.

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-01 Thread Anton Makeev


 On 01 Apr 2015, at 00:41, Stephen Kelly steve...@gmail.com wrote:
 
 Anton Makeev wrote:
 
 The problem with the following format:
 ---
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 —
 Is that it’s incomplete and cannot be used directly:
 * include directories list misses compiler-defined search paths
 
 CMake gathers those during configure time, and we can add them to the 
 metadata too.

However appealing it is, I don’t think it’s a good option, since it'll make 
generation
phase way longer, since files and targets may have different command line 
options and
CMake will need to call compiler for all of them. We’d rather do it lazily when 
times come
to process the given file having specialized caches on IDE side.


 * quote include directories are not distinguishable
 
 CMake does not currently generate -iquote as a result of a user writing an 
 include_directories() call. 
 
 Are you asking for that as an option? That would be an independent feature 
 request. I'd suggest filing it in the tracker.
 Otherwise, the only way CMake currently would pass such options to the 
 compiler is if the user specified them in CMAKE_CXX_FLAGS or 
 add_compile_options() or similar. 

Yes, I mean that user may easily have -iquote directly passed as command line
option and the IDE need to know that.

 CMake does not attempt to parse CMAKE_CXX_FLAGS for things which look like 
 include directories. That variable is just an opaque string of data from the 
 point of view of CMake. It is not clear to me whether you are suggesting 
 that CMake should do that. If you do want to suggest that, please file an 
 issue in the tracker.
 Similarly, CMake does not attempt to analyse the meaning of options 
 specified in add_compile_options(). Again, it's not clear to me whether you 
 are suggesting a new feature for CMake. If you are, please file it in the 
 tracker.

No, I don’t suggest CMake should do that, sorry for the confusion.
I was just explaining why we need the whole compiler-specific command line, 
instead of 
a list of separate flags and include directories  - we’ll pass that command 
line 
as is (actually with some post-processing) to the compiler to collect the actual
defines, header search paths etc. (some examples below)

The current proposal (with having separate include_directories, 
compile_definitions, compile_flags) will force the IDE to assemble the 
compiler-specific 
command line itself that is quite prone to errors and will require constant 
maintenance
since additional options may appear (e.g. include_directories may some day have 
a
flag to indicate whether it “” or )

 To avoid misunderstandings, please post snippets of CMake code together with 
 a description of what information you want CMake to determine from the 
 snippet and what metadata it should generate.

Not sure what you are asking, I hope I made it more clear with previous 
explanation.

 * compiler definition do not include compiler-defined(built-in)
 definitions etc.
 
 CMake does not currently gather those during configure, as far as I know. 
 That could be added though.

It'd rather not, since the set may be different for the specific files.

 That is, to have the complete list of include directories and compiler
 definitions, the IDE will have to call the compiler anyways. And here is
 where it will need the actual compiler’s command line.
 
 Should we had the full list of include directories (split into groups
 like: , quote, and frameworks) and the list of all compiler defines, we
 would not need the compiler command-line at all.
 
 I think I understand what you're saying. However, it would help if you 
 related what you write to existing CMake features and snippets of CMake 
 code. I assume that you are medium-to-very familiar with CMake features 
 already, given what you are working on?

Given the details above, please let me know, if you still need them,
I’ll try to collect the specific cases that we support from our test suite.

 Though I don’t this it’s a
 way to go for CMake generation, since the compiler should be called for
 every unique command-line flags and undoubtedly make generation way
 longer. 
 
 It's not completely clear to me what you are suggesting here. Maybe you know 
 something I don't know.
 
 Can you be clear about what you are suggesting CMake would have to do for 
 each unique command line? I know you are not proposing that CMake should do 
 it (you said the IDE should do it lazily instead), but what action are you 
 actually talking about? What should the IDE do lazily instead of CMake doing 
 it? Invoke the compiler to determine the compiler-defined/built-in include 
 directories and definitions? They will always be the same, right? Or are you 
 talking about the -I- GCC option here and how that clears the built-ins? 
 Please be more specific.

We currently reconstruct the command line from the 
'build_dir/CMakeFiles/Target1.dir/flags.make’ file

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-01 Thread Anton Makeev

 On 01 Apr 2015, at 00:17, Stephen Kelly steve...@gmail.com wrote:
 
 Anton Makeev wrote:
 
 It comes from CMAKE_CONFIGURATION_TYPES, if project defined such a
 variable. With default of Debug/Release/RelWithDebInfo/MinSizeRel
 
 Ok.
 
 So does this mean you have a parser for the existing CMakeCache.txt format 
 already?

Yes, at the moment we parse lots of various internal files created by Makefiles 
generator,
and CMakeCache is one of them.

Not only do we reed it, we also have the UI that allows user to interact and  
the cached values. 
That means we’ll have CMakeCache parser anyways and do not need these values
in metadata.json file, as it was mentioned in parallel discussion.


Anton Makeev
JetBrains
http://www.jetbrains.com
Develop with pleasure!



smime.p7s
Description: S/MIME cryptographic signature
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-01 Thread Stephen Kelly

Anton Makeev wrote:

 
 On 01 Apr 2015, at 00:41, Stephen Kelly
 steve...@gmail.com wrote:
 
 Anton Makeev wrote:
 
 The problem with the following format:
 ---
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 —
 Is that it’s incomplete and cannot be used directly:
 * include directories list misses compiler-defined search paths
 
 CMake gathers those during configure time, and we can add them to the
 metadata too.
 
 However appealing it is, I don’t think it’s a good option,

Ok, I understand now, because the used built-in defines are different more 
often than the built-in include directories (-iquote and -I- are rarely used 
in my experience, but -std=, -ansi, -fPIC which all cause defines to be 
added, are common).

 We’d rather do it lazily when times come to process the given
 file having specialized caches on IDE side.

Ok. Do you currently use compile-commands.json for this?

 Are you asking for that as an option? That would be an independent
 feature request. I'd suggest filing it in the tracker.
 Otherwise, the only way CMake currently would pass such options to the
 compiler is if the user specified them in CMAKE_CXX_FLAGS or
 add_compile_options() or similar.
 
 Yes, I mean that user may easily have -iquote directly passed as command
 line option and the IDE need to know that.

 (e.g. include_directories may some day have a flag to indicate whether it
 “” or )

Ok, I filed 

 http://public.kitware.com/Bug/view.php?id=15491

Please add any additional information you have, or any design ideas for how 
the command interface should be.



 CMake does not attempt to parse CMAKE_CXX_FLAGS for things which look
 like include directories. That variable is just an opaque string of data
 from the point of view of CMake. It is not clear to me whether you are
 suggesting that CMake should do that. If you do want to suggest that,
 please file an issue in the tracker.
 Similarly, CMake does not attempt to analyse the meaning of options
 specified in add_compile_options(). Again, it's not clear to me whether
 you are suggesting a new feature for CMake. If you are, please file it in
 the tracker.
 
 No, I don’t suggest CMake should do that, sorry for the confusion.
 I was just explaining why we need the whole compiler-specific command
 line, instead of
 a list of separate flags and include directories

Ok, thanks. I get it now :).

 The current proposal (with having separate include_directories,
 compile_definitions, compile_flags) will force the IDE to assemble the
 compiler-specific command line itself that is quite prone to errors and
 will require constant maintenance since additional options may appear
 (e.g. include_directories may some day have a flag to indicate whether it
 “” or )

I agree. However, there are two separate use-cases to address.

1) Get information from CMake so that the IDE can understand the C++
2) Get information to understand the CMake code.
 
You are talking about the first, but I think the second is useful too and an 
IDE could offer features related only to the CMake code (ie unrelated to the 
c++ code) based on that information (such as 'Add include directory' 
action).

So, in the end, we probably need tooling to be able to get the information 
in both forms. Ie, as an array of directories, and as a string of compiler 
command line options.

We already have the latter in the form of compile-commands.json. One of the 
reasons I wonder if you are using that already is to know whether 

1) That is already a good enough solution 
or 
2) The new metadata file should contain similar information

I'm wondering if we can learn anything from your experience with the 
existing file.

 * compiler definition do not include compiler-defined(built-in)
 definitions etc.
 
 CMake does not currently gather those during configure, as far as I know.
 That could be added though.
 
 It'd rather not, since the set may be different for the specific files.

Yes, you're right I think.

 Given the details above, please let me know, if you still need them,
 I’ll try to collect the specific cases that we support from our test
 suite.

I think the details you provided above are enough for understanding, but 
getting some cases from your test suite relating to this stuff would indeed 
be interesting, I expect.

 We currently reconstruct the command line from the
 'build_dir/CMakeFiles/Target1.dir/flags.make’ file for each target,

I see. There are several disadvantages to that, among them being that it 
ties you to makefiles (until you duplicate your work to support Ninja), and 
the structure of those files should not be considered 'stable'. CMake should 
be able to change the way it generates the buildsystem-specific files in any 
release.


 We then add additional flags like -dD -M etc. to get the necessary
 information from the compiler and call (compiler is actually called only
 once for the unique set of options with cashing etc.). And we have

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-01 Thread Stephen Kelly

Tobias Hunger wrote:

 On Wed, Apr 1, 2015 at 12:41 AM, Stephen Kelly steve...@gmail.com wrote:
 Anton Makeev wrote:

 The problem with the following format:
 ---
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 —
 Is that it’s incomplete and cannot be used directly:
 * include directories list misses compiler-defined search paths

 CMake gathers those during configure time, and we can add them to the
 metadata too.
 
 That is not necessary for QtC. Feel free to add this information if
 somebody else needs it.

Ok. I have some ideas of how this could be useful, so I'll see about adding 
it later.

 snip
 
 * compiler definition do not include compiler-defined(built-in)
 definitions etc.

 CMake does not currently gather those during configure, as far as I know.
 That could be added though.
 
 That is not necessary for QtC. Feel free to add this information if
 somebody else needs it.

Ok, I think the conclusion is that this is not useful as the built-in 
defines are very likely to change with different compile flags, which are 
very likely to be set in the buildsystem *after* CMake would have checked 
them from the compiler. 

So, I don't think this will be added.

 Creator will parse the compile_flags anyway, so it does not matter to
 me whether some defines/includes end up being hidden in there, but I
 would prefer to get as much of the information separately if possible.
 So please keep the include_directories and compile_defines that cmake
 knows about separate. That is so much nicer to parse than some command
 line, especially when escaping starts to get necessary.

Right. That sounds like an argument for the kind of 'duplication for 
different use cases/approaches' I just proposed:

 
http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/12658/focus=12879

 That is, to have the complete list of include directories and compiler
 definitions, the IDE will have to call the compiler anyways. And here is
 where it will need the actual compiler’s command line.

 Should we had the full list of include directories (split into groups
 like: , quote, and frameworks) and the list of all compiler defines,
 we would not need the compiler command-line at all.
 
 We do need to know the compiler *binary* (/usr/bin/gcc), but not
 necessarily the complete command line that is run (/usr/bin/gcc -c
 -Dsomething -I/some/path something.cpp something.o). We do of course
 need to be sure that we have all the information to generate that
 command line in the form of compile_flags, compile_definitions,
 include_directories and whichever other flags may be relevant.

Ok.

 If you want to log the compiler-defined defines into the Json file,
 then cmake obviously needs to query the compiler for those.
 
 These defines are heavily dependent on the command line flags you pass
 (e.g. -std=c++11 will set some other defines than -std=c++98). There
 is a long list of options that change the defines set by the compiler.

Right, this is what I temporarily forgot :).

 So that means cmake would need to run the compiler once for each
 possible set of options passed to the compiler during the later build
 steps, so that it can put this information into the meta-data file.
 That seems a bit excessive to me,

Agreed.

 even though creator could probably
 use that information if you did provide it. Currently our
 projectmanagers just query for the compiler defines themselves. They
 have to as currently none of the supported build system provides this
 information.

Right, because it's impractical for the buildsystem to provide it, as you 
wrote.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-01 Thread Stephen Kelly

Anton Makeev wrote:

 Not only do we reed it, we also have the UI that allows user to interact
 and  the cached values. That means we’ll have CMakeCache parser anyways
 and do not need these values in metadata.json file, as it was mentioned in
 parallel discussion.

Yes, that's what I thought. I favor documenting the existing format instead 
of adding a new json format for it. I filed 

 http://public.kitware.com/Bug/view.php?id=15490

to track the idea.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-04-01 Thread David Cole via cmake-developers

I favor documenting the existing format, too.

Note added to issue with my vote.


D


On Wed, Apr 1, 2015 at 4:26 PM, Stephen Kelly steve...@gmail.com wrote:
 Anton Makeev wrote:

 Not only do we reed it, we also have the UI that allows user to interact
 and  the cached values. That means we’ll have CMakeCache parser anyways
 and do not need these values in metadata.json file, as it was mentioned in
 parallel discussion.

 Yes, that's what I thought. I favor documenting the existing format instead
 of adding a new json format for it. I filed

  http://public.kitware.com/Bug/view.php?id=15490

 to track the idea.

 Thanks,

 Steve.


 --

 Powered by www.kitware.com

 Please keep messages on-topic and check the CMake FAQ at: 
 http://www.cmake.org/Wiki/CMake_FAQ

 Kitware offers various services to support the CMake community. For more 
 information on each offering, please visit:

 CMake Support: http://cmake.org/cmake/help/support.html
 CMake Consulting: http://cmake.org/cmake/help/consulting.html
 CMake Training Courses: http://cmake.org/cmake/help/training.html

 Visit other Kitware open-source projects at 
 http://www.kitware.com/opensource/opensource.html

 Follow this link to subscribe/unsubscribe:
 http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-31 Thread Stephen Kelly

Anton Makeev wrote:

 The problem with the following format:
 ---
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 —
 Is that it’s incomplete and cannot be used directly:
 * include directories list misses compiler-defined search paths

CMake gathers those during configure time, and we can add them to the 
metadata too.

 * quote include directories are not distinguishable

CMake does not currently generate -iquote as a result of a user writing an 
include_directories() call. 

Are you asking for that as an option? That would be an independent feature 
request. I'd suggest filing it in the tracker.

Otherwise, the only way CMake currently would pass such options to the 
compiler is if the user specified them in CMAKE_CXX_FLAGS or 
add_compile_options() or similar. 

CMake does not attempt to parse CMAKE_CXX_FLAGS for things which look like 
include directories. That variable is just an opaque string of data from the 
point of view of CMake. It is not clear to me whether you are suggesting 
that CMake should do that. If you do want to suggest that, please file an 
issue in the tracker.

Similarly, CMake does not attempt to analyse the meaning of options 
specified in add_compile_options(). Again, it's not clear to me whether you 
are suggesting a new feature for CMake. If you are, please file it in the 
tracker.

To avoid misunderstandings, please post snippets of CMake code together with 
a description of what information you want CMake to determine from the 
snippet and what metadata it should generate.

 * compiler definition do not include compiler-defined(built-in)
 definitions etc.

CMake does not currently gather those during configure, as far as I know. 
That could be added though.

 That is, to have the complete list of include directories and compiler
 definitions, the IDE will have to call the compiler anyways. And here is
 where it will need the actual compiler’s command line.
 
 Should we had the full list of include directories (split into groups
 like: , quote, and frameworks) and the list of all compiler defines, we
 would not need the compiler command-line at all.

I think I understand what you're saying. However, it would help if you 
related what you write to existing CMake features and snippets of CMake 
code. I assume that you are medium-to-very familiar with CMake features 
already, given what you are working on?

 we’ll need to call the compiler with the correct
 command-line to get some more info like specific compiler features, but I
 guess CMake could do it instead of the IDE).

I don't know what you are refering to, but if you want CMake to record more 
compiler features than what it already does:

 http://www.cmake.org/cmake/help/v3.1/manual/cmake-compile-features.7.html

then please file an issue in the tracker.

The information gathered using the above feature can be added to the 
metadata file.

 Though I don’t this it’s a
 way to go for CMake generation, since the compiler should be called for
 every unique command-line flags and undoubtedly make generation way
 longer. 

It's not completely clear to me what you are suggesting here. Maybe you know 
something I don't know.

Can you be clear about what you are suggesting CMake would have to do for 
each unique command line? I know you are not proposing that CMake should do 
it (you said the IDE should do it lazily instead), but what action are you 
actually talking about? What should the IDE do lazily instead of CMake doing 
it? Invoke the compiler to determine the compiler-defined/built-in include 
directories and definitions? They will always be the same, right? Or are you 
talking about the -I- GCC option here and how that clears the built-ins? 
Please be more specific.

Thanks!

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-30 Thread Anton Makeev

On 21 Mar 2015, at 09:41, Stephen Kelly steve...@gmail.com wrote:

Anton Makeev wrote:

The other thing that seems troubling to me is that since file, target,
language compiler options are split into different parts of metadata, the
IDE need to know exactly how to assemble them back into the compiler’s
command line (e.g. what flags go first file’s or language’s), duplicating
CMake's logic that may be different from version to version and from
compiler to compiler. The exact command line is needed to get the actual
and precise defines, include search paths etc. from the compiler.

Yes. I previously proposed the lang_compile_command to contain information
about how to built it:

http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html#optional-properties

Thanks for pointing, I missed the purpose of that property.
If we are after a compact form, this should work, I guess.

However, I think it might be better to generate something similar to what is
currently generated in compile-commands.json into cmake-metadata.json. That
is, we would generate (in some context)

include_directories : [/foo, /opt]
compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
compile_command: -c -DDEF=\Foo\ -DOTHER_DEF=1 -I/foo -I/opt

So, compile_command contains approximately what you can currently get from
compile-commands.json.

The other properties contain things which are specifically known to be
include directories or compile definitions, as javascript arrays. These
properties are obviously redundant information, so I wonder if they should
be generated at all? Is the compile command I wrote above easy to parse? Or
is it sufficiently difficult to parse that this redundant information should
be provided?

Agree, I don’t see any point in having this redundant info. I’d better not have
it to avoid confusion:
the include_directories lists is incomplete (it doesn’t contain
compiler-defined search paths) and should not be used for any purpose.

This would be really helpful indeed, currently, we have to introspect
CMakeLists.txt files in order to find the most probably place where new
files should be placed (works only in basic cases now). And being able to
do so correctly is also crucial for refactoring (e.g. extract class).

Given the backtrace, you can navigate up the scope from the most recent
frame to get out of any functions, macros or loops. You can then add a
target_sources() line directly after that.

That algorithm will work for every case (not just basic cases) as far as I
can tell and is available with CMake 3.1.

OK, it’s definitely better than nothing, so we can start with the target
backtraces.

* I didn't document the location or directory. I'm not clear on whether
it is supposed to be the build location, or the install location(s!),
or all of those.

It would be useful, though, to have a location of generated files for each
target: in case metadata misses some information (and I think it won’t
cover every possible need anytime soon), IDE will be able to get if from
generated makefiles.

Yes, we can at least provide the build location in an obvious way. We can
discuss install locations eventually.

* I don't generate 'dependencies' (actually the list of files which the
buildsystem re-generation depends on) as Aleix did, because there is no
well-defined usefulness for that list yet.

As Tobias pointed, we at least need to know what files are the part of
CMake project, that is, the list of all CMakeLists.txt and *.cmake files,
used for generation (ideally, including missing ones, since in that case
IDE could be able to tell when missing file is created and refresh the
project)

As I wrote to Tobias, I'm apprehensive about this, and it would require
other work to make cmake parallel safe first.

I think if the IDE does not have focus it should not be running 'cmake .' on
my behalf. I think if the IDE newly gets focus you can maybe run 'cmake .'
at *that* point (after the user is done with their rebase or whatever). That
doesn't require giving you a list of files to watch. Maybe I'm missing
something though.

Agree, the IDE should not do any automatic stuff, when user doesn’t expect it.
Though, the use-case here is quite different:
Consider a project that have 'add_directory(missing_dir)’ in one of the
CMakeLists files.
If IDE were aware that missing_dir/CMakeLists.txt is a required file, it might
then automatically
trigger update, when the user creates this file inside the IDE (not
externally).
Or it may even have a quick-fix ‘Create missing CMakeList.txt’.

Also, files updated and created during VCS (called from the IDE) should be
automatically recognized, of course.

* Some more information from project() may be relevant, but it's not
clear
yet. We will likely know more when we have decided the file format and
generated some 'interesting' metadata files.

Project name, list of the

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-30 Thread Anton Makeev

 On 24 Mar 2015, at 00:54, Stephen Kelly steve...@gmail.com wrote:
 
 Tobias Hunger wrote:
 
 How about include_directories, compile_definitions and compile_flags?
 
 So something along the lines of:
 
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 
 Quoting Anton:
 
 Anton Makeev wrote:
 The other thing that seems troubling to me is that since file, target,
 language compiler options are split into different parts of metadata, the
 IDE need to know exactly how to assemble them back into the compiler’s
 command line (e.g. what flags go first file’s or language’s), duplicating
 CMake's logic that may be different from version to version and from
 compiler to compiler. The exact command line is needed to get the actual
 and precise defines, include search paths etc. from the compiler.
 
 Maybe he can chime in with more. I don't really know why the entire command 
 line is needed instead of separate include_directories and 
 compile_definitions arrays as you and I suggested. Perhaps because that 
 would not include -fPIC for example, which causes __PIC__ to be defined.
 
 Another item of note is that CMake does not know the compile flags as a 
 sequential container of individual flags currently, but it knows them as a 
 string (that's also why it appears as a string in my generated json 
 currently).

The problem with the following format:
---
include_directories : [/foo, /opt]
compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
compile_flags: [ -c ]
—
Is that it’s incomplete and cannot be used directly:
* include directories list misses compiler-defined search paths
* quote include directories are not distinguishable
* compiler definition do not include compiler-defined(built-in) definitions
etc.

That is, to have the complete list of include directories and compiler 
definitions, the IDE will have to call the compiler anyways.
And here is where it will need the actual compiler’s command line.

Should we had the full list of include directories (split into groups like: , 
quote, and frameworks) and the list of all compiler defines, we would not need 
the compiler command-line at all.
(It’s actually not completely true, since we’ll need to call the compiler with 
the correct command-line to get some more info like specific compiler features, 
but I guess CMake could do it instead of the IDE).
Though I don’t this it’s a way to go for CMake generation, since the compiler 
should be called for every unique command-line flags and undoubtedly make 
generation way longer.
That’s why I think it’s up to the IDE to (lazily) call the compiler when this 
information is necessary.

Does it make sense or maybe I’m missing something?

 Parsing things is always error prone. Is that -D for definitions or /D?
 
 As we can define the format, we could always generate -D and define that as 
 the answer to your question.
 
 Anyway, let's see if Anton can explain more about the need for the exact 
 command line and whether my suggestion of duplicating the 
 include_directories and compile_definitions is fine.
 
 The compiler flags are definitely needed though. They are used to e.g.
 decide which dialect of a language are used.
 
 This would keep parsing simple and will also provide all the
 information we need.
 
 linker_flags might also be interesting...
 
 Yes, something like this is also in the goals for this design.
 
 Thanks,
 
 Steve.



Anton Makeev
JetBrains
http://www.jetbrains.com
Develop with pleasure!



smime.p7s
Description: S/MIME cryptographic signature
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-30 Thread Anton Makeev


 On 30 Mar 2015, at 21:21, Alexander Neundorf neund...@kde.org wrote:
 
 On Monday, March 30, 2015 19:04:10 Anton Makeev wrote:
 On 24 Mar 2015, at 00:54, Stephen Kelly steve...@gmail.com wrote:
 
 Tobias Hunger wrote:
 How about include_directories, compile_definitions and compile_flags?
 
 So something along the lines of:
 
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 
 Quoting Anton:
 
 Anton Makeev wrote:
 The other thing that seems troubling to me is that since file, target,
 language compiler options are split into different parts of metadata, the
 IDE need to know exactly how to assemble them back into the compiler’s
 command line (e.g. what flags go first file’s or language’s), duplicating
 CMake's logic that may be different from version to version and from
 compiler to compiler. The exact command line is needed to get the actual
 and precise defines, include search paths etc. from the compiler.
 
 Maybe he can chime in with more. I don't really know why the entire
 command
 line is needed instead of separate include_directories and
 compile_definitions arrays as you and I suggested. Perhaps because that
 would not include -fPIC for example, which causes __PIC__ to be defined.
 
 Another item of note is that CMake does not know the compile flags as a
 sequential container of individual flags currently, but it knows them as a
 string (that's also why it appears as a string in my generated json
 currently).
 
 The problem with the following format:
 ---
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 —
 Is that it’s incomplete and cannot be used directly:
 * include directories list misses compiler-defined search paths
 * quote include directories are not distinguishable
 * compiler definition do not include compiler-defined(built-in) definitions
 etc.
 
 
 Code for detecting the built-in include dirs and definitions is in 
 CMakeExtraGeneratorDetermineCompilerMacrosAndIncludeDirs.cmake. This is used 
 currently by CMakeFindCodeBlocks.cmake and CMakeFindEclipseCDT4.cmake, i.e. 
 the respective extra-generators, and the information is generated into the 
 eclipse/codeblocks project files.
 This could be used here too.
 
 I don't really understand your second point. Can you elaborate ?

Sure,
'some compilers’ distinguish between “include” and include search paths, gcc 
has -iquote and -I- parameters to specify “include” search path:
https://gcc.gnu.org/onlinedocs/cpp/Search-Path.html

Distinguishing them is necessary to correctly resolve headers inside the IDE.
Additionally, there is frameworks search paths (os OS X), and it’s very 
important to differentiate them as well.

Here is clang’ list just for the reference:
---
#include ... search starts here:
#include ... search starts here:
 
/Applications/Dev/Xcode-6.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/include
 
/Applications/Dev/Xcode-6.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
---

Cheers,
Anton Makeev
JetBrains
http://www.jetbrains.com
Develop with pleasure!



smime.p7s
Description: S/MIME cryptographic signature
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-30 Thread Tobias Hunger

On Mon, Mar 30, 2015 at 7:04 PM, Anton Makeev
anton.mak...@jetbrains.com wrote:
 Another item of note is that CMake does not know the compile flags as a
 sequential container of individual flags currently, but it knows them as a
 string (that's also why it appears as a string in my generated json
 currently).

 The problem with the following format:
 ---
 include_directories : [/foo, /opt]
 compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
 compile_flags: [ -c ]
 —
 Is that it’s incomplete and cannot be used directly:
 * include directories list misses compiler-defined search paths
 * quote include directories are not distinguishable
 * compiler definition do not include compiler-defined(built-in) definitions
 etc.

 That is, to have the complete list of include directories and compiler 
 definitions, the IDE will have to call the compiler anyways.
 And here is where it will need the actual compiler’s command line.

Oh, yes, we do need the path to the compiler to be run, too.

I think we can combine the include_directories, compile definitions
and flags ourselves though when that is needed for us to run the
compiler. I do not see the need to have the full command line that
would be run for each and every file, which is how I read full
compiler command line.

 Should we had the full list of include directories (split into groups like: 
 , quote, and frameworks) and the list of all compiler defines, we would not 
 need the compiler command-line at all.

We need the compiler binary anyway:-)

Best Regards,
Tobias
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-29 Thread Stephen Kelly

Stephen Kelly wrote:

 I expect to require a few iterations to figure out what the metadata files
 should contain in the end.

Just FYI, I have not forgotten about this and am working on an expanded 
design to cover the reported needs.

Thanks, 

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-23 Thread Stephen Kelly

Tobias Hunger wrote:

 and
 which kit (think set of compiler, Qt, some other settings) were used
 to in the built in that directory.

 Is this hypothetical? Currently I can add a compiler location and a Qt
 location in QtCreator, and I can create a kit and give it a name and
 partcular compiler and Qt. However, the information I configure there and
 the kit that I choose is not passed to cmake when creator invokes that,
 right? CMake will do its own determination of the compiler to use, and
 will find Qt on its own, right? Unless the user manually sets those as
 arguments in the 'Run CMake' dialog page.
 
 We do that for qmake projects at this time.
 
 Nothing listed here is needed to get the level of cmake support we
 have in Creator right *now*: That works fine already after all. I want
 something better:-) So I bring up all the things I can think of to get
 cmake to the same level that we currently have for qmake.

Right, I see.

 Or are you thinking of scenarios such as 'the user creates a build
 outside of Creator and then tries to open it in creator'. Given that 'a
 kit' is something internal to Creator, there is no way CMake could tell
 you the kit in that scenario. Is that obvious or am I missing something?
 I think the best thing you can do there is find out which compiler was
 used, and what the value of Qt5Core_DIR is, and map that to things you
 know about available kits to find the match if there is one.
 
 Yes, that is the same we do for qmake projects. In fact we just create
 a new kit for configurations we had not seen before.

Right. In the case that Creator imports a cmake build dir for a project 
which uses Qt 5, and which is not already configured in Creator, you would 
need the path to qmake for the Qt used by it, because that is what Creator 
uses as the key for a particular Qt (or that used to be the case). 

By the way, just as another tip, in that case the reliable way to get the 
corresponding qmake would be to get the Qt5Core_DIR from the cache in the 
just-found build dir (via the json or whatever), and write a file to a temp 
dir with the content:

 cmake_minimum_required(VERSION 2.8)
 project(find_qmake NONE)
 find_package(Qt5Core REQUIRED)
 get_target_property(loc Qt5::qmake LOCATION)
 message(QMAKE_LOCATION: \${loc}\)

and then run cmake on it and parse the result:

 stephen@hal:/tmp/cmake/build$ cmake .. \
   -DQt5Core_DIR=/usr/lib/x86_64-linux-gnu/cmake/Qt5Core/
 QMAKE_LOCATION: /usr/lib/x86_64-linux-gnu/qt5/bin/qmake
 -- Configuring done
 -- Generating done
 -- Build files have been written to: /tmp/cmake/build

 stephen@hal:/tmp/cmake/build$ cmake .. \
   -DQt5Core_DIR=/opt/qt/binary/Qt5.2.1/5.2.1/gcc_64/lib/cmake/Qt5Core
 QMAKE_LOCATION: /opt/qt/binary/Qt5.2.1/5.2.1/gcc_64/bin/qmake
 -- Configuring done
 -- Generating done
 -- Build files have been written to: /tmp/cmake/build

 I only want the IDE to re-generate when I'm actually working in the IDE.
 If I edit any file and hit Ctrl+B, the IDE runs 'cmake --build .' and
 cmake regenerates if needed at that point. It's 'ok' if the project tree
 is technically 'stale' after I edit a file and before I hit Ctrl+B. It's
 'ok' in the sense that the alternative is 'worse' as far as my opinion
 goes. You also have the option of running 'cmake .' any time any file is
 saved in the IDE, if that improves the experience, but I strongly believe
 you shouldn't be watching the files for outside changes and re-generating
 in response.
 
 That might be ok for you, but we do get a lot of bug reports whenever
 we have the project tree not update directly. 

'Update directly' means 'I save a file in the IDE and the target treeview 
updates', right? 

Quoting myself above: running 'cmake .' any time any file is saved in the 
IDE.

 So it is *not* ok for a
 large number of users, particularly those that come from a proprietary
 IDE that comes with a built-in build system.

For my curiosity, what scenario are you talking about which is not covered 
by triggering regeneration on 'saving in the IDE' and 'newly getting focus'?

 We do *not* update creator while it is not in use. 

Fantastic!

 When the window
 gets focus again we go Things have changed outside Creator, do you
 want to update?.

Great.

 That's not to say we won't 'give you the rope'. But before we would do
 that, cmake would have to be defensive against concurrent runs in the
 same build dir.
 
 A simple lock file should suffice. I would be surprised if that was
 not there already, but I never checked.

Yes, I think you're right. I haven't checked yet either, but either way it 
is necessary.

 The key difference is that my proposal is only requested values, so is
 minimal.
 
 Yes. But then you need to add cmake -E list_cache that lists all
 values. Or how can I find out which values are there in a project?

Indeed.

 In the end it is probably easier to either put all that stuff into the
 json file or to declare CMakeCache.txt to be a stable read-only file
 for the

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-22 Thread Tobias Hunger

On Sat, Mar 21, 2015 at 9:41 AM, Stephen Kelly steve...@gmail.com wrote:
Anton Makeev wrote:

Yes. I previously proposed the lang_compile_command to contain information
about how to built it:

http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html#optional-properties

However, I think it might be better to generate something similar to what is
currently generated in compile-commands.json into cmake-metadata.json. That
is, we would generate (in some context)

include_directories : [/foo, /opt]
compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
compile_command: -c -DDEF=\Foo\ -DOTHER_DEF=1 -I/foo -I/opt

So, compile_command contains approximately what you can currently get from
compile-commands.json.

How about include_directories, compile_definitions and compile_flags?

So something along the lines of:

include_directories : [/foo, /opt]
compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
compile_flags: [ -c ]

Parsing things is always error prone. Is that -D for definitions or /D?

The compiler flags are definitely needed though. They are used to e.g.
decide which dialect of a language are used.

This would keep parsing simple and will also provide all the
information we need.

linker_flags might also be interesting...

Best Regards,
Tobias
--

Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-21 Thread Stephen Kelly

Tobias Hunger wrote:

 Hi Anton,
 
 you raised some good points, all of which I agree with:-)
 
 On Thu, Mar 19, 2015 at 10:18 AM, Anton Makeev
 anton.mak...@jetbrains.com wrote:
 * If it is useful to preprocess/compile/assemble individual files from
  IDEs, as made possible by the Makefiles and Ninja generators, we'll need
  to design that.
 
http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12429


 This is definitely useful, but I’m not sure what kind of information
 needed here,
 as I see it, since we already know the files in the project, we can tell
 make/ninja to simply compile it.
 
 You asked me to use cmake --build, so ideally that would be cmake
 --build . /full/file/path and ideally it would work with all
 generators without magic in the IDE:-)

It is also orthogonal to the metadata of the build itself and can be 
designed separately.

I filed

 http://public.kitware.com/Bug/view.php?id=15465

if you want to engage in the design or implementation of that.


 
 Since I assume that not all build systems will allow to build
 individual files, you might want to add a flag
 can_build_individual_files or similar to the metadata that a
 generator can use to flag the IDE on whether the generated build
 system can build individual files or not. Then the IDE can hide that
 option if it is not applicable.

Yes. If it's not possible for xcodebuild/VS, then such a property can be 
added. Noted in 

 http://public.kitware.com/Bug/view.php?id=15462

 I would also love to see subprojects:-) CMake allows for them, doesn't it?

I don't know what 'subprojects' means to you.

 An additional though: here only the 'project information' aspect is
 discussed; though, to be fully machine-frienly, cmake should be able to
 also generate parseable output (error reports etc), provide the progress,
 etc. So, just to mull over, probably the discussed design should consider
 such future direction.
 
 Yes, that would be great, but I do not see how cmake can do that: It
 delegates the actual build to external tools.

Anton is talking about output of cmake itself afaik.

 So in the end during a build we will always have to deal with whatever
 output the generated buildsystem throws at us:-/ This again somewhat
 limits the usefulness of allowing the user to pick whichever generator
 they want: Some will just loose some or all the build issues.

Only allow the user to choose a generator for whose make_program you have a 
parser for.

Thanks,

Steve.



-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-21 Thread Stephen Kelly

Anton Makeev wrote:

Yes. I previously proposed the lang_compile_command to contain information
about how to built it:

http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html#optional-properties

However, I think it might be better to generate something similar to what is
currently generated in compile-commands.json into cmake-metadata.json. That
is, we would generate (in some context)

include_directories : [/foo, /opt]
compile_definitions : [DEF=\Foo\, OTHER_DEF=1]
compile_command: -c -DDEF=\Foo\ -DOTHER_DEF=1 -I/foo -I/opt

So, compile_command contains approximately what you can currently get from
compile-commands.json.

I have no idea if such compile_command can be generated for VS or Xcode,
or if constructing such compile commands is done internally by those tools.
So, this may not be a portable solution anyway.

Given the backtrace, you can navigate up the scope from the most recent
frame to get out of any functions, macros or loops. You can then add a
target_sources() line directly after that.

That algorithm will work for every case (not just basic cases) as far as I
can tell and is available with CMake 3.1.

* I didn't document the location or directory. I'm not clear on whether
it is supposed to be the build location, or the install location(s!),
or all of those.

Yes, we can at least provide the build location in an obvious way. We can
discuss install locations eventually.

* I don't generate 'dependencies' (actually the list of files which the
buildsystem re-generation depends on) as Aleix did, because there is no
well-defined usefulness for that list yet.

As I wrote to Tobias, I'm apprehensive about this, and it would require
other work to make cmake parallel safe first.

* Some more information from project() may be relevant, but it's not
clear
yet. We will likely know more when we have decided the file format and
generated some 'interesting' metadata files.

Project name, list of the configurations are most needed ones.
We also use CMAKE_lang_SOURCEFILE_EXTENSIONS to determine if a given
file is potentially source file or not.

As CMake already knows which files are 'object sources', the metadata will
provide that. Also, the lang extensions is not enough. See the unit test I
created and in particular the compiled_as_cxx.c file.

This has already been discussed but I give our usage scenario:

in CLion we retrieve the list of all build types (aka configurations,
Debug, Release etc)

From where do you currently retrieve this list? I guess you look at all
cache keys named

CMAKE_.*_FLAGS_(.*)

and list the matches?

and then generate project using Makefiles generator
for each of them. This is necessary because of several reasons: 1) To be
able to correctly build language model, we need to know, when a file is
used in several configurations, which means, it's compiler settings

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-21 Thread Stephen Kelly

Tobias Hunger wrote:

 On Wed, Mar 11, 2015 at 11:10 AM, Stephen Kelly
 steve...@gmail.com wrote:
 * Make it possible for editors/IDEs to allow specifying the configuration
   at build-time, where the IDE has that feature, and where a multi-config
   generator is available.  That is, QtCreator provides user-interface for
   choosing debug/release config to build.  Currently it can't offer that
   when using cmake, because it only allows the use of Makefile or Ninja
   generators, in order to make use of the C::B file.  QtCreator should be
   able to use the Xcode or Visual Studio generators, generate the
   metadata file(s), and invoke `cmake --build . --config ${CONFIG}` as
   appropriate.  Eclipse, Sublime and other editors have similar abilities
   to invoke config-specific builds after generation.
 
 We can offer debug/release even with different build directories in
 Creator. Those are just labels on the build configuration that
 includes the build directory and the configuration to be set up in
 that build.

Interesting. Is that already the case in Creator?

 For the other build systems it is completely safe to just nuke any
 build directory and creator will re-initialize it on the next build.
 That is unfortunately not the case with cmake at this time though: If
 you nuke a cmake build directory then creator will be completely lost
 about how to regenerate the configuration again.

Ah, is this why you asked before for the command line options passed to 
cmake before?

Why is it safe with other buildsystems but not cmake to nuke the build dir?

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-19 Thread Tobias Hunger

Hi Anton,

you raised some good points, all of which I agree with:-)

On Thu, Mar 19, 2015 at 10:18 AM, Anton Makeev
anton.mak...@jetbrains.com wrote:
 * If it is useful to preprocess/compile/assemble individual files from
  IDEs, as made possible by the Makefiles and Ninja generators, we'll need
  to design that.
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12429


 This is definitely useful, but I’m not sure what kind of information needed
 here,
 as I see it, since we already know the files in the project, we can tell
 make/ninja to simply compile it.

You asked me to use cmake --build, so ideally that would be cmake
--build . /full/file/path and ideally it would work with all
generators without magic in the IDE:-)

Since I assume that not all build systems will allow to build
individual files, you might want to add a flag
can_build_individual_files or similar to the metadata that a
generator can use to flag the IDE on whether the generated build
system can build individual files or not. Then the IDE can hide that
option if it is not applicable.

 * Some more information from project() may be relevant, but it's not clear
  yet. We will likely know more when we have decided the file format and
  generated some 'interesting' metadata files.


 Project name, list of the configurations are most needed ones.
 We also use CMAKE_lang_SOURCEFILE_EXTENSIONS to determine if a given file
 is potentially source file or not.

I would also love to see subprojects:-) CMake allows for them, doesn't it?

Will you still need CMAKE_lang_SOURCEFILE_EXTENSIONS when you can
get access to the list of sources as above where each group of files
is flagged with the language, etc.?

snip

 As a side note, it seems more natural to me to have one json file with one
 or several configurations listed, providing that there is also shared
 project info that should be in that files.
 something like that:

 project: ProjectName
 configurations: {
  {
name: Debug
targets: {...}
  },
  {
   name: Release
targets: {...}
  }
  ...
 }

I'd actually prefer having a configurations key with the list of the
configurations an object applies to.

That can be optional if the object is relevant to all configurations
(default in single-configuration generators;-).

This key would then need to be applicable to targets as well as the
group of source files.

So that would be something like this:

{
  project: TestProject,
  targets: [ {
name: Debug only,
configurations: [ Debug ],
type: EXECUTABLE,
sources: [ ... ]
  },
  {
name: Release only,
configurations: [ Release ],
type: EXECUTABLE,
sources: [ ... ]
  },
  {
name: Run Tests,
sources: sources: {
language: C++,
defines: FOO=1,

sources: [ {
  type: sources,
  name: Critical source files,
  configurations: [ Debug ],
  include_paths: /some/path,
  files: [ /full/path/a.cpp, /full/path/b.cpp ]
},
{
  type: headers,
  files: [ /full/path/a.h ]
},
{
  type: generated,
  files: [ /full/path/config.h ]
},
{
  type: sources,
  name: Tests,
  include_paths: /some/path,
  files: [ /full/path/a.cpp, /full/path/b.cpp, /full/path/test.cpp ]
} ]
  }
}

That would most likely avoid quite a bit of duplication in the file(s).

  This also means that conditions don't need to be added inside the metadata
  file for configurations in order to show files 'excluded' by being part of
  a non-active configuration.  The code implementing discovery of excluded
  files is in the generate-metadata branch in my clone.

Yeap, that would be a convenient property for me, too.

snip

 * Generating metadata only (without generating buildsystem files) is not
  currently in scope.  This was requested several times, but it is not
  clear why.


 It’s simply to be able to get this the information as quickly as possible.
 I’m not sure which part is most slow, but, say, InsightToolKit 4.5
 (http://www.itk.org/Wiki/ITK/Source), generates in couple of minutes. The
 regeneration, even when nothing was changes, a few dozens of seconds.

 Plus, we’d prefer being able to open the project without any questions to
 user, e.g. not asking, which generator he/she prefers. If we generate using
 ‘wrong’ default generator we’ll need to regenerate everything again when
 user decides to change it.

 Another benefit of skipping actual generation is possibly better error
 recoverability.
 That is, some generators may fail here and there if the project is
 incorrectly configured (e.g. source files are missing). Skipping the
 generation phase will (probably) help getting the project metadata even in
 that case.

 But anyway, it seems a little outside of the scope of the discussion.

Very good points that I fully agree with.

 * How much information does tooling need about installation?  Targets
  can use different include directories and compile definitions in their
  install

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-19 Thread Tobias Hunger

Hi Stephen,

On Thu, Mar 19, 2015 at 2:22 AM, Stephen Kelly steve...@gmail.com wrote:
 * In Target Properties/Conditional Properties and optional
 properties: Would it make sense to have a list of target_files, each
 with a filepath and a type? That would be more similar to the targets
 which also are a list with name/type.

 What semantics would the 'type' have? I don't really understand what you
 propose here.

See the examples with the sources below:

sources: [ {
language: C++,
type: sources,
defines: FOO=1,
include_paths: /usr/include,
files: [ /full/path/a.cpp, /full/path/b.cpp ]
},
{
language: C++,
type: headers,
files: [ /full/path/a.h ]
},
{
language: C++,
type: generated,
files: [ /full/path/config.h ]
}]

The type is used to distinguish between the different kinds of files.
Your proposal uses different keys for all the different kinds of
files, my is to use the same key for all sources and distinguish
between them by giving a type.

snip

 These configurations are only relevant to generators that can support
 e.g. having debug and release builds in the same build directory or
 did I misunderstand this?

 Yes, your understanding is correct.

 How are we supposed to handle a set of files for these configurations?

 Yes, that's how I see it working...

 How can we notice when one gets added and removed?

 I'm not experienced enough with those generators to know. I'm not even
 certain whether it is possible to add to the list of configurations for
 those generators, or if the list is hardcoded.

 Something to look into or get information from someone else on this list if
 they know.

 I think in the worst case, you'd watch the CMakeCache.txt and read the
 CMAKE_CONFIGURATION_TYPES from there. I know you want to care about only one
 file, but maybe two is not so bad :), and there is potentially other stuff
 in the cache which might be useful, and which might not lend itself to the
 json metadata file. We'll have to see.

So we are back to parsing random files in internal formats again:-)

Could you add have a configuration list key in the top level of each
file that lists all available configurations (including itself)? That
might be the lowest overhead solution that does not require parsing
extra files that I can think of right now.

I definitely will not hardcode cmake generator-A has configurations X
and Y while generator-A supports configurations X, Z and D into the
cmake plugin. I do prefer not to support configurations at all to
doing that.

snip

 * Make it more easy for an IDE to support actions such as 'remove file
   from the project', which requires removing it from the CMakeLists.txt
   at the appropriate place, and 'add new file/class to target', which
   involves adding code to the CMakeLists.txt file at the appropriate
   place.  Most likely the easiest way to do the latter is using the
   target_sources() command, and to support the former, the location of
   the declaration of the target, and all target_sources() calls would
   need to be recorded.  Even that is not enough because of transitive
   consumption of source files through the link interface, but that is
   likely irrelevant.

 That would be the icing on the cake, but getting a full description of
 the project is what I care for at this point.

 Can you expand on what 'a full description of the project' means to you? Do
 you mean something different than 'how each object is compiled'?

When you open a project in Creator we offer to import existing
builds (at least for qmake-based projects, none of the other build
systems supports this yet). That is very convenient when you blew away
the .user file we put into the sources directory with all the
information on how you built stuff where.

What we do there is to check the default project build location for
folders that contain a build of the current sources.

To do so we need to know which code was built in that directory and
which kit (think set of compiler, Qt, some other settings) were used
to in the built in that directory. We also need to figure out the
exact configuration (parameters passed to the configure step of the
build system) that was built there.

Since this is a feature our qmake users love I would also like to make
that possible with cmake. It would be great if I could get most
answers from the json file.

 * I don't generate 'dependencies' (actually the list of files which the
   buildsystem re-generation depends on) as Aleix did, because there is no
   well-defined usefulness for that list yet.

 How is the IDE supposed to know that it needs to re-run cmake then? If
 a file changes that may very well change the list of files that the
 IDE needs to display, so re-running cmake only at the next build is
 not going to work.

 Ok, I didn't realize that's the scenario we're talking about for that
 content. Are you saying that you would

   Set up a

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-16 Thread Stephen Kelly

Aleix Pol wrote:

 By 'in scope' you mean 'used by the primary CMakeLists.txt files via
 include() or find_package()' or something like that? OK, let's see if
 that's the best source of that data or if we should add something new
 internally in cmake.
 By in scope I mean whether a variable or a target defined in such files.

I see.

 [4] Because cmake is really slow and you often want to have the
 metadata file updated often, every time one of the cmake files in the
 project change.

 OK, maybe we can add something for that, but is it the generation stage
 that is the slow part? How much time would we gain by doing this?
 Well, I'm unsure what's the best way to go about it. I understand that
 given the json file will be executed after all the processing of the
 sources it won't help much if we just skip the generation. Also maybe
 it's possible to optimize the configure process all together.

Yes, if a 'metadata only' mode can be shown to make sense, it can be added 
in the future.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-16 Thread Stephen Kelly

Nils Gladitz wrote:

 On 03/11/2015 11:10 AM, Stephen Kelly wrote:
 I'm starting to gather requirements and make sure the feature is
 well designed to satisfy the needs we're already aware of, and fits with
 the features CMake currently has.
 
 Source file groups (as in defined by source_group()) and target folders
 (as in the FOLDER target property) may be of interest to IDEs to layout
 target and source file hierarchies.

Indeed. I don't have enough experience with VS to comment much on this. I've 
added a commit containing some todos and added it there.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-14 Thread Stephen Kelly

On Saturday, March 14, 2015, Aleix Pol aleix...@kde.org wrote:
On Wed, Mar 11, 2015 at 11:10 AM, Stephen Kelly steve...@gmail.com
wrote:
Hi,

Following from the thread here:

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12394

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12650

I'm starting to gather requirements and make sure the feature is
well designed to satisfy the needs we're already aware of, and fits with
the features CMake currently has.

The aim is to generate a structured file containing metadata relating the
buildsystem.

To help with completing the design of this feature, I've written
documentation (documentation driven design), and a unit test
containing a CMakeLists.txt file which exercises many modern CMake
features in the generate-metadata branch in my clone.

Mostly the design I propose can be read in the documentation I wrote. I'm
interested in any feedback.

https://gitorious.org/cmake/steveires-cmake/source/generate-metadata:Tests/Metadata/CMakeLists.txt

http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html

I expect to require a few iterations to figure out what the metadata
files
should contain in the end. Note that there are already some differences
between my design and Aleix's implementation, such as that my design
proposes one metadata file per config. There are also some things
missing like location, because it is not yet clear to me whether build
or install locations are needed etc.

The content of the metadata file is determined by the build properties,
and
is necessarily similar to the compile-related content created when
generating the actual buildsystem. It additionally contains information
about the output locations of build artifacts and information relating to
the cmake description itself.

Goals include:

* Make it possible for IDEs to access the compile-related information for
autocompletion and code navigation etc purposes.

* Remove the need for IDEs to parse generated Makefiles or Ninja files to
access compile-related information. The structure of those files is
not
'stable', while the content of the metadata file is stable.
http://thread.gmane.org/gmane.comp.programming.tools.cmake.user/48412

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11081

* Remove the need for users to create a new build directory and new build
in order to use or switch IDEs. QtCreator requires that
the C::B 'extra generator is used as it parses compile information from
that. Other 'extra generators' such as for eclipse, sublime, kate etc
also require fresh/new build directories, although the actual
buildsystem
they create is identical (assuming using all Makefile based or
all Ninja based 'extra generators')

* Make it possible to write a plugin for the editors/IDEs such as sublime
which consumes the metadata file and invokes the build using whatever
buildsystem the user already has a build directory for, instead of
writing an 'extra generator' and maintaining it in the cmake repo.
http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/9004

* Make it possible for editors/IDEs to allow specifying the configuration
at build-time, where the IDE has that feature, and where a multi-config
generator is available. That is, QtCreator provides user-interface for
choosing debug/release config to build. Currently it can't offer that
when using cmake, because it only allows the use of Makefile or Ninja
generators, in order to make use of the C::B file. QtCreator should be
able to use the Xcode or Visual Studio generators, generate the
metadata
file(s), and invoke `cmake --build . --config ${CONFIG}` as
appropriate. Eclipse, Sublime and other editors have similar abilities
to invoke config-specific builds after generation.

* Provide a list of 'build targets', which can be listed and invoked in
IDE/editor user interface. Build targets for all linked binaries
and utilties are provided. The tooling is expected to perform
filtering
on the target types to show only executables and utilities for
execution, for example.

* Provide a list of source files per target per type of source file, eg
object sources, header files, generated files, files excluded from the
active configuration/platform/compiler, non-compiled files.

* Make it more easy for an IDE to support actions such as 'remove file
from the project', which requires removing it from the CMakeLists.txt
at the appropriate place, and 'add new file/class to target', which
involves adding code to the CMakeLists.txt file at the appropriate
place. Most likely the easiest way to do the latter is using the
target_sources() command, and to support the former, the location of
the declaration of the target, and all target_sources() calls would
need to be recorded. Even that is

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-14 Thread Aleix Pol

On Sat, Mar 14, 2015 at 10:02 AM, Stephen Kelly steve...@gmail.com wrote:

On Saturday, March 14, 2015, Aleix Pol aleix...@kde.org wrote:
On Wed, Mar 11, 2015 at 11:10 AM, Stephen Kelly steve...@gmail.com
wrote:
Hi,

Following from the thread here:

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12394

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12650

I'm starting to gather requirements and make sure the feature is
well designed to satisfy the needs we're already aware of, and fits with
the features CMake currently has.

The aim is to generate a structured file containing metadata relating the
buildsystem.

Mostly the design I propose can be read in the documentation I wrote. I'm
interested in any feedback.

https://gitorious.org/cmake/steveires-cmake/source/generate-metadata:Tests/Metadata/CMakeLists.txt

http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html

Goals include:

* Make it possible for IDEs to access the compile-related information for
autocompletion and code navigation etc purposes.

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11081

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-13 Thread Aleix Pol

On Wed, Mar 11, 2015 at 11:10 AM, Stephen Kelly steve...@gmail.com wrote:
Hi,

Following from the thread here:

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12394

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12650

I'm starting to gather requirements and make sure the feature is
well designed to satisfy the needs we're already aware of, and fits with
the features CMake currently has.

The aim is to generate a structured file containing metadata relating the
buildsystem.

Mostly the design I propose can be read in the documentation I wrote. I'm
interested in any feedback.

https://gitorious.org/cmake/steveires-cmake/source/generate-metadata:Tests/Metadata/CMakeLists.txt
http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.html

I expect to require a few iterations to figure out what the metadata files
should contain in the end. Note that there are already some differences
between my design and Aleix's implementation, such as that my design
proposes one metadata file per config. There are also some things
missing like location, because it is not yet clear to me whether build
or install locations are needed etc.

The content of the metadata file is determined by the build properties, and
is necessarily similar to the compile-related content created when
generating the actual buildsystem. It additionally contains information
about the output locations of build artifacts and information relating to
the cmake description itself.

Goals include:

* Make it possible for IDEs to access the compile-related information for
autocompletion and code navigation etc purposes.

* Remove the need for IDEs to parse generated Makefiles or Ninja files to
access compile-related information. The structure of those files is not
'stable', while the content of the metadata file is stable.
http://thread.gmane.org/gmane.comp.programming.tools.cmake.user/48412

http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=11081

* Remove the need for users to create a new build directory and new build
in order to use or switch IDEs. QtCreator requires that
the C::B 'extra generator is used as it parses compile information from
that. Other 'extra generators' such as for eclipse, sublime, kate etc
also require fresh/new build directories, although the actual buildsystem
they create is identical (assuming using all Makefile based or
all Ninja based 'extra generators')

* Make it possible for editors/IDEs to allow specifying the configuration
at build-time, where the IDE has that feature, and where a multi-config
generator is available. That is, QtCreator provides user-interface for
choosing debug/release config to build. Currently it can't offer that
when using cmake, because it only allows the use of Makefile or Ninja
generators, in order to make use of the C::B file. QtCreator should be
able to use the Xcode or Visual Studio generators, generate the metadata
file(s), and invoke `cmake --build . --config ${CONFIG}` as
appropriate. Eclipse, Sublime and other editors have similar abilities
to invoke config-specific builds after generation.

* Provide a list of 'build targets', which can be listed and invoked in
IDE/editor user interface. Build targets for all linked binaries
and utilties are provided. The tooling is expected to perform filtering
on the target types to show only executables and utilities for
execution, for example.

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-13 Thread Nils Gladitz


On 03/11/2015 11:10 AM, Stephen Kelly wrote:

I'm starting to gather requirements and make sure the feature is
well designed to satisfy the needs we're already aware of, and fits with
the features CMake currently has.


Source file groups (as in defined by source_group()) and target folders 
(as in the FOLDER target property) may be of interest to IDEs to layout 
target and source file hierarchies.


Nils

--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-13 Thread Stephen Kelly

On 03/11/2015 11:10 AM, Stephen Kelly wrote:
 Hi,

 Following from the thread here:

  
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12394
  
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus=12650


The discussion in that thread is causing people confusion (eg regarding
an obsolete approach to config and CMAKE_BUILD_TYPE). Please discontinue
the discussion there. My branch is the way forward.

I extended my branch to contain not only test and documentation but also
an implementation and a generated file from my computer, so that you can
read it/try it and give feedback. Obviously the branch needs to be
broken up into multiple commits, but that something to do when this is
not a prototype anymore. We are still in a design discussion.

I pushed the implementation only because I know you guys won't discuss
the design of the feature on its own and you want to have an
implementation asap. However, this is still a design discussion, and all
of the questions in my previous mail are still valid.

Thanks,

Steve.

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Generating buildsystem metadata from CMake

2015-03-11 Thread Alexander Neundorf

On Wednesday, March 11, 2015 11:10:30 Stephen Kelly wrote:
 Hi,
 
 Following from the thread here:
 
  http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focu
 s=12394
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focu
 s=12650
 
 I'm starting to gather requirements and make sure the feature is
 well designed to satisfy the needs we're already aware of, and fits with
 the features CMake currently has.
 
 The aim is to generate a structured file containing metadata relating the
 buildsystem.
 
 To help with completing the design of this feature, I've written
 documentation (documentation driven design), and a unit test
 containing a CMakeLists.txt file which exercises many modern CMake
 features in the generate-metadata branch in my clone.
 
 Mostly the design I propose can be read in the documentation I wrote. I'm
 interested in any feedback.
 
  https://gitorious.org/cmake/steveires-cmake/source/generate-metadata:Tests/
 Metadata/CMakeLists.txt
 http://www.steveire.com/cmake-future/manual/cmake-metadata-generation.7.htm
 l
 
 I expect to require a few iterations to figure out what the metadata files
 should contain in the end.  Note that there are already some differences
 between my design and Aleix's implementation, such as that my design
 proposes one metadata file per config. There are also some things
 missing like location, because it is not yet clear to me whether build
 or install locations are needed etc.
 
 The content of the metadata file is determined by the build properties, and
 is necessarily similar to the compile-related content created when
 generating the actual buildsystem.  It additionally contains information
 about the output locations of build artifacts and information relating to
 the cmake description itself.
 
 Goals include:
 
 * Make it possible for IDEs to access the compile-related information for
   autocompletion and code navigation etc purposes.
 
 * Remove the need for IDEs to parse generated Makefiles or Ninja files to
   access compile-related information.  The structure of those files is not
   'stable', while the content of the metadata file is stable.
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.user/48412
 
 http://thread.gmane.org/gmane.comp.programming.tools.cmake.devel/10711/focus
 =11081
 
 * Remove the need for users to create a new build directory and new build
   in order to use or switch IDEs.  QtCreator requires that
   the C::B 'extra generator is used as it parses compile information from
   that.  Other 'extra generators' such as for eclipse, sublime, kate etc
   also require fresh/new build directories, although the actual buildsystem
   they create is identical (assuming using all Makefile based or
   all Ninja based 'extra generators')

From my POV, the json metadata is just another extra generator, but one 
which can be always enabled additionally to other generators.
You say that users don't need new build directories for switching IDEs, but 
this is only under the assumption that the different IDEs will actually start 
to use this new file format.
See, users can use the C::B format and switch between CodeBlocks and 
QtCreator. If KDevelop and Kate would get plugins to read the C::B format 
(which is technically entirely possible) their users also would't have to 
create a new build directory.
So instead of adding generators to cmake which generate what the IDEs expect, 
this is a change, now cmake will generate a file and the IDEs are expected to 
follow.

My 2 cents
Alex

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] Generating buildsystem metadata from CMake

2015-03-11 Thread Stephen Kelly

Hi,