Re: [cmake-developers] Using CMake as a library from Python

[Charles Huet]

I took the time to make it work again, so I pushed it here:
https://github.com/packadal/CMake/tree/cmake-python

The whole branch is ugly as can be, and because I started with an old CMake
and recently rebased, the python abstraction might be bloated already, but
it works in the nominal case, and it would be pretty easy to add most of
the CMake basic functionalities.

I am not used to writing python, so it might not be idiomatic either, but
hey, it works as a simple proof of concept, and it's all I wanted to do for
now :)

Le lun. 4 janv. 2016 à 08:41, Charles Huet <charles.h...@gmail.com> a
écrit :

> Hi,
>
> Because of the above, I worry that you are basing your work on an old
>> version of CMake. As of April 2015 cmState is used as the interface to the
>> cache.
>
>
> Yeah, that is sadly right. I wanted to rebase on master before publishing,
> thinking it should not be too hard, and I have lots to re-do, mostly
> understand how things are done now.
>
> I did something similar some years ago with QML instead of python, so it
>> sounds interesting to me.
>
>
> I'm trying to be as declarative as possible, because really like how
> readable simple QML programs are, and I think it would be perfect for a
> buildsystem.
>
> My guess is that you are using python to instantiate a cmAddLibraryCommand
>> and then executing it.
>
>
> Actually, I'm directly using the cmMakefile, because I did not want to
> wrap all the commands, and it seemed backwards to me to wrap them.
>
> Even much of cmMakefile shouldn't be used by a new language. Instead
>> certain
>> parts of cmMakefile should be extracted as other classes
>> (cmVariableExpander
>> which should have the ExpandVariablesInString and ConfigureString stuff,
>> cmMessenger for the IssueMessage stuff, some other class for the
>> CompileFeature stuff etc). Then cmMakefile would be just about executing
>> and
>> scoping the CMake language. A new language would not need that, but would
>> use the refactored extracted classes.
>
>
> Ah, this is very interesting, thanks.
>
>
> Having said all that, Brad favors Lua I believe, and he favors a different
>> approach (which no one is working on as far as I know) to adding a new
>> language. So wait to hear from him to know whether it is something that
>> would be upstreamable.
>
>
> Have any details on the approach in question ? SWIG would allow for Lua
> bindings as easily, but I don't think having multiple languages would be a
> good idea.
> I went with Python because I'm familiar with it and have already written
> bindings for it with SWIG. Also, buildbot is written in python and it could
> provide a really interesting integration I think.
>
> I would guide/support you in refactoring cmake as needed. The refactoring
>> part would definitely be upstreamable. I would very much like to see a
>> proof
>> of concept alternative language even if that wasn't upstreamed. It would
>> prove that another language is possible, and that's one of the steps to
>> replacing the current cmake language I think.
>
>
> I will need to work on it to make it work again with master, but I'll try
> and do this soon.
>
> Here is what my test POC looked like for generating a simple shared
> library:
>
> #!/usr/bin/env python
>> # -*- coding: utf-8 -*-
>> import cmake
>> cmake.init("Ninja", "/media/dev/src/cmaketest",
>> "/media/dev/build/cmaketest")
>> myProject = cmake.Project("MyTestProject")
>> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
>> cmake.generate()
>
>
> Thanks a lot for your comments, I'm happy to see this was not just a dumb
> idea, and that others have thought about it.
>
> I'll update with the github as soon as I get it working.
>
> Best
>
>
> Le lun. 4 janv. 2016 à 01:16, Stephen Kelly <steve...@gmail.com> a écrit :
>
>> Charles Huet wrote:
>>
>> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
>> > cannot be used from python (it check for the existence of a
>> CMakeLists.txt
>> > file, which does not exist in this scenario) and the cache variables it
>> > sets seem to be necessary.
>>
>> Because of the above, I worry that you are basing your work on an old
>> version of CMake. As of April 2015 cmState is used as the interface to the
>> cache.
>>
>>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13
>>
>> Also note that the C++ interfaces in CMake are not stable. In the last
>> year
>> they have been changed utterly and that will continue

Re: [cmake-developers] Using CMake as a library from Python

[Charles Huet]

Yes, this is POC-only.

I felt too lazy to make an argparse object and properly perform these tasks.
Obviously, the generator and source dir should be arguments.
The build dir should be the execution dir of the script, to mimic CMake
behavior.

This should not be difficult, but as I said I wanted first to get this
disussion started.

Le lun. 4 janv. 2016 à 11:54, Tamás Kenéz <tamas.ke...@gmail.com> a écrit :

> Charles,
>
> I'd like to comment on a minor issue:
>
> CMakeLists.txt files don't specify user-specific data like source and
> build dirs (for a reason) yet your POC listfile contains this line:
>
> cmake.init("Ninja", "/media/dev/src/cmaketest",
> "/media/dev/build/cmaketest")
>
> which sets the source and build directories. Is this some temporary
> solution for the POC only?
>
> Tamas
>
> On Mon, Jan 4, 2016 at 8:41 AM, Charles Huet <charles.h...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Because of the above, I worry that you are basing your work on an old
>>> version of CMake. As of April 2015 cmState is used as the interface to
>>> the
>>> cache.
>>
>>
>> Yeah, that is sadly right. I wanted to rebase on master before
>> publishing, thinking it should not be too hard, and I have lots to re-do,
>> mostly understand how things are done now.
>>
>> I did something similar some years ago with QML instead of python, so it
>>> sounds interesting to me.
>>
>>
>> I'm trying to be as declarative as possible, because really like how
>> readable simple QML programs are, and I think it would be perfect for a
>> buildsystem.
>>
>> My guess is that you are using python to instantiate a cmAddLibraryCommand
>>> and then executing it.
>>
>>
>> Actually, I'm directly using the cmMakefile, because I did not want to
>> wrap all the commands, and it seemed backwards to me to wrap them.
>>
>> Even much of cmMakefile shouldn't be used by a new language. Instead
>>> certain
>>> parts of cmMakefile should be extracted as other classes
>>> (cmVariableExpander
>>> which should have the ExpandVariablesInString and ConfigureString stuff,
>>> cmMessenger for the IssueMessage stuff, some other class for the
>>> CompileFeature stuff etc). Then cmMakefile would be just about executing
>>> and
>>> scoping the CMake language. A new language would not need that, but would
>>> use the refactored extracted classes.
>>
>>
>> Ah, this is very interesting, thanks.
>>
>>
>> Having said all that, Brad favors Lua I believe, and he favors a different
>>> approach (which no one is working on as far as I know) to adding a new
>>> language. So wait to hear from him to know whether it is something that
>>> would be upstreamable.
>>
>>
>> Have any details on the approach in question ? SWIG would allow for Lua
>> bindings as easily, but I don't think having multiple languages would be a
>> good idea.
>> I went with Python because I'm familiar with it and have already written
>> bindings for it with SWIG. Also, buildbot is written in python and it could
>> provide a really interesting integration I think.
>>
>> I would guide/support you in refactoring cmake as needed. The refactoring
>>> part would definitely be upstreamable. I would very much like to see a
>>> proof
>>> of concept alternative language even if that wasn't upstreamed. It would
>>> prove that another language is possible, and that's one of the steps to
>>> replacing the current cmake language I think.
>>
>>
>> I will need to work on it to make it work again with master, but I'll try
>> and do this soon.
>>
>> Here is what my test POC looked like for generating a simple shared
>> library:
>>
>> #!/usr/bin/env python
>>> # -*- coding: utf-8 -*-
>>> import cmake
>>> cmake.init("Ninja", "/media/dev/src/cmaketest",
>>> "/media/dev/build/cmaketest")
>>> myProject = cmake.Project("MyTestProject")
>>> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
>>> cmake.generate()
>>
>>
>> Thanks a lot for your comments, I'm happy to see this was not just a dumb
>> idea, and that others have thought about it.
>>
>> I'll update with the github as soon as I get it working.
>>
>> Best
>>
>>
>> Le lun. 4 janv. 2016 à 01:16, Stephen Kelly <steve...@gmail.com> a
>> écrit :
>>
>>> Charles Huet wrote:
>

Re: [cmake-developers] Using CMake as a library from Python

[Charles Huet]

Hi,

Because of the above, I worry that you are basing your work on an old
> version of CMake. As of April 2015 cmState is used as the interface to the
> cache.


Yeah, that is sadly right. I wanted to rebase on master before publishing,
thinking it should not be too hard, and I have lots to re-do, mostly
understand how things are done now.

I did something similar some years ago with QML instead of python, so it
> sounds interesting to me.


I'm trying to be as declarative as possible, because really like how
readable simple QML programs are, and I think it would be perfect for a
buildsystem.

My guess is that you are using python to instantiate a cmAddLibraryCommand
> and then executing it.


Actually, I'm directly using the cmMakefile, because I did not want to wrap
all the commands, and it seemed backwards to me to wrap them.

Even much of cmMakefile shouldn't be used by a new language. Instead certain
> parts of cmMakefile should be extracted as other classes
> (cmVariableExpander
> which should have the ExpandVariablesInString and ConfigureString stuff,
> cmMessenger for the IssueMessage stuff, some other class for the
> CompileFeature stuff etc). Then cmMakefile would be just about executing
> and
> scoping the CMake language. A new language would not need that, but would
> use the refactored extracted classes.


Ah, this is very interesting, thanks.


Having said all that, Brad favors Lua I believe, and he favors a different
> approach (which no one is working on as far as I know) to adding a new
> language. So wait to hear from him to know whether it is something that
> would be upstreamable.


Have any details on the approach in question ? SWIG would allow for Lua
bindings as easily, but I don't think having multiple languages would be a
good idea.
I went with Python because I'm familiar with it and have already written
bindings for it with SWIG. Also, buildbot is written in python and it could
provide a really interesting integration I think.

I would guide/support you in refactoring cmake as needed. The refactoring
> part would definitely be upstreamable. I would very much like to see a
> proof
> of concept alternative language even if that wasn't upstreamed. It would
> prove that another language is possible, and that's one of the steps to
> replacing the current cmake language I think.


I will need to work on it to make it work again with master, but I'll try
and do this soon.

Here is what my test POC looked like for generating a simple shared library:

#!/usr/bin/env python
> # -*- coding: utf-8 -*-
> import cmake
> cmake.init("Ninja", "/media/dev/src/cmaketest",
> "/media/dev/build/cmaketest")
> myProject = cmake.Project("MyTestProject")
> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
> cmake.generate()


Thanks a lot for your comments, I'm happy to see this was not just a dumb
idea, and that others have thought about it.

I'll update with the github as soon as I get it working.

Best


Le lun. 4 janv. 2016 à 01:16, Stephen Kelly <steve...@gmail.com> a écrit :

> Charles Huet wrote:
>
> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
> > cannot be used from python (it check for the existence of a
> CMakeLists.txt
> > file, which does not exist in this scenario) and the cache variables it
> > sets seem to be necessary.
>
> Because of the above, I worry that you are basing your work on an old
> version of CMake. As of April 2015 cmState is used as the interface to the
> cache.
>
>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13
>
> Also note that the C++ interfaces in CMake are not stable. In the last year
> they have been changed utterly and that will continue to happen without
> notice. I'm sure you know this, but I thought I'd say it anyway.
>
> > I will try to make a layer of abstraction on top of the python bindings
> > before publishing this work on github, in order to show the syntactic
> > sugar python can provide, but I will publish this before if anybody is
> > interested in working on this.
>
> I would be interested in seeing it.
>
> > Now, does anyone beside me think this is a good idea ?
>
> I did something similar some years ago with QML instead of python, so it
> sounds interesting to me.
>
> In fact, one of the reasons for introducing cmState and doing all the
> refactoring I did in cmake was to make it possible to cleanly replace the
> language, some day.
>
> My guess is that you are using python to instantiate a cmAddLibraryCommand
> and then executing it. I think a better approach would be to leave all the
> cm*Command classes behind as they carry a lot of backward compatibility
> baggage and policies which a new l

[cmake-developers] Using CMake as a library from Python

[Charles Huet]

Hi,

I'd like to have a discussion about using CMake as a library from python,
and by that I mean not writing CMakeLists.txt files, but python code that
then uses the CMake generators and Find*.cmake in order to leverage both
the power of CMake and Python.

First, a bit of background on the idea itself.

I've used CMake since 2010 in different projects at work, and I love it.
I'm the CMake guy at work, because most of the colleagues cannot be
bothered with it. In fact I highly prefer doing the CMake maintenance
myself, as most of the colleagues do not make any effort in trying to apply
best practices of to factorize the code (which they do with the C++ code).

When talking about CMake, the first reaction most of them have is about the
language itself, because they don't know it, don't like some of the idioms,
and have trouble accomplishing what they want with it.

Nobody argues that CMake is awesome at generating either Visual Studio
solutions or Unix Makefiles or whatever, but the language is a high barrier
of entry.

I have to admit that doing seemingly simple stuff can be a bit confusing in
cmake, e.g. checking if the value of a variable is greater than another.


Thus I thought that using a language that is well-known and has a closer
syntax to what we are used to (not to mention we use python quite a bit for
custom scripts) would greatly ease the maintenance of our buildsystem.

This would allow for a few things that are not straightforward in cmake,
such as functions that return a value.

I prototyped around this idea these last weeks (a few minutes or hours at a
time) and got it in a proof of concept state, where I cant do a add_library
in python.

I think this would be a great step forward for writing buildsystems, as
lots of tasks would be greatly simplified. From the top of my mind:
* reading files
* writing files
* downloading files
* copying files (CMake's copy is not POSIX-compliant, as I learned recently
when copying symbolic links)

Also, the syntax could be made Object-Oriented very easily, as the python
bindings will need a layer of abstraction.

Now, for the technical realisation

My prototype required a few modifications into CMake itself:
* I added cmSystemTools::SetCMakeRoot, as the system root cannot be
determined automatically using the current methods
* cmLinkItem's typedef of std::string was made public because swig has
trouble with it. There may be a better way to solve the problem, but this
was the quickest for a POC
* cmCacheManager::AddCacheEntry was made public, as cmake::Configure cannot
be used from python (it check for the existence of a CMakeLists.txt file,
which does not exist in this scenario) and the cache variables it sets seem
to be necessary.

A cmake.i swig interface file, and a python file that does the
initialization of CMake, and I have a simple library compiled.

I will try to make a layer of abstraction on top of the python bindings
before publishing this work on github, in order to show the syntactic sugar
python can provide, but I will publish this before if anybody is interested
in working on this.


Now, does anyone beside me think this is a good idea ?


Cheers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CMake alternative language

[Charles Huet]

g that this change.
>
> * CMake's procedural/imperative design is good as the main entry point to
>   configuration of a project.  It can do system introspection, file
> generation,
>   etc.  I'm not proposing that this change.
>
> * Many projects build elaborate macro/function systems in the CMake
> language
>   in order to end up with a declarative specification listing the actual
>   source files, dependencies, and usage requirements.  I'd like to offer
>   an alternative to this.
>
> * Integration with IDEs is currently based on one-way generation (VS IDE
>   projects, Xcode projects, CodeBlocks, etc.).  Editing the project build
>   specification requires editing CMake code directly because IDEs cannot
>   easily pierce CMake's procedural/imperative specification:
>
> https://cmake.org/pipermail/cmake-developers/2016-January/027386.html
>
>   I'd like to improve this by *optionally* moving part of the specification
>   to a (stateless) declarative format that IDEs can load/edit/save
> directly.
>
> Specific responses follow.
>
>
> 
>
> On 01/11/2016 12:24 PM, Charles Huet wrote:
> > I think these goals aim towards a faster configure, and the ability to
> > only partly reconfigure, right?
>
> Yes.
>
> > I know I am largely biased by the project I work on, but I do not see how
> > parallel evaluation woud be a huge benefit.
> [snip]
> > And how would that work with CMakeLists that affect their parent scope ?
>
> Evaluation of the imperative language is currently serial for reasons like
> this, which is why I said it would take semantic changes to enable parallel
> evaluation. This is not the main point of my proposal so I'd rather not
> get bogged down in the details of this part of the discussion.
>
> >> Ideally most of the specification (sources, libraries, executables,
> etc.)
> >> should be in a pure format that can be evaluated without side effects
> (e.g.
> >> declarative or functional).
> >
> > I'm not sure I understand how this could be done without losing a lot of
> > what CMake offers, such as copying or generating files.
>
> I'm not proposing dropping the current imperative capabilities.
>
> > I'm leaning towards a declarative approach as it is quite easy to learn
> > (since you declare objects, and every C++ programmer I know is familiar
> > with those)
>
> Yes.
>
> > It seems you are leaning towards pure functional, but I do not see how
> > this would work with the current way CMake handles variables and scope,
> > could you elaborate ?
>
> While declarative may get us most of the way, advanced users may wish to
> hook in to generation-time evaluation. A clean way to do that would be
> to specify a function within the declared values. It would not have to
> be in a functional language as long as it has no access to anything other
> than the inputs passed to it during evaluation.
>
> I mentioned "functional" mostly as an example of a specification whose
> evaluation is free of side effects.
>
> > To clarify, only the following lines should be considered when looking
> at the POC.
> >> myProject=cmake.Project("MyTestProject")
> >> myProject.targets=[cmake.SharedLibrary("testLibrary",["lib.cxx"])]
>
> Yes, this is the kind of stuff that can be in a declarative format.
>
> > It seems you have in mind to write a new CMake language.
>
> No, at most a new specification format that can be used for IDE
> integration.
> If some kind of user-coded function were included in the specification it
> should certainly be in an existing language.
>
> > Maybe I should take my POC further
>
> I think implementation even of a POC is premature at this point.  We
> should explore the design space further.
>
> > CMake's own buildsystem seems like a good testing ground for this, but
> > it is a little too big for a first go, do you know of a small CMake-based
> > project that would be better suited ?
>
> Maybe you could find something in our test suite.
>
> > I don't have a clear view of what a pure functional CMake would look
> like,
> > but if you give me some mock code, I could give a try at bringing some
> pure
> > functional language up to the level of my POC and we could use it as a
> more
> > concrete discussion support.
>
> I have no prototype (nor substantial time to spend on design myself) but
> I've imagined a declarative format in a well-known syntax (e.g. JSON or one
> of the lightweight human-friendly choices).  If generate-time functionality
> is 

Re: [cmake-developers] CMake alternative language

[Charles Huet]

I don't think this is a dumb question, actually this is part of the problem
I think would be resolved whit a new language.

The barrier of entry to using CMake is too high in my opinion, and I think
using an existing language would lower it *a lot*.

Thanks for sharing :)


Le mer. 13 janv. 2016 à 10:59, yann suisini <yannsuis...@gmail.com> a
écrit :

> Hi,
>
> I'm a new user of CMake, but I just want to express my newcomer point of
> view.
> Honestly , I can feel the power of CMAKE, but it's a real pain to learn ...
> I'm using CMAKE for an embedded platform with a non GNU compiler , ant at
> the end the CMAKE description is longer than the one I built directly in
> Ninja.
> I had to write a python script to parse my eclipse project xml to create a
> list of sources files usable by CMAKE.
> The first thing I thought was: why this is not a part of cmake ? And the
> second thing was : why not using the scripting power of an existing
> language like Python(or other one)
> and add CMAKE as a framework / library ?
> Probably a dumb question ! :)
>
> Yann
>
> 2016-01-13 10:34 GMT+01:00 Charles Huet <charles.h...@gmail.com>:
>
>> Hi,
>>
>> > * There is a lot of code out there in the current CMake language so I
>> do not
>>   think it is realistic to drop it.  I'm not proposing that this change.
>>
>> I am. (more below)
>>
>> > * Many projects build elaborate macro/function systems in the CMake
>> language
>>   in order to end up with a declarative specification listing the actual
>>   source files, dependencies, and usage requirements.  I'd like to offer
>>   an alternative to this.
>>
>> In my experience, most of the elaborate macros/functions come either from
>> a misunderstanding of some of CMake's internals (scope, link debug/release
>> libs, etc) or to circumvent some shortcoming of the CMake language (e.g. no
>> return value for functions).
>>
>> >  I'd like to improve this by *optionally* moving part of the
>> specification
>>   to a (stateless) declarative format that IDEs can load/edit/save
>> directly
>>
>> Split the buildsystem in two different languages ? Would the declarative
>> part be in a different file ?
>> Also, the declarative part in my opinion must take advantage of the
>> language.
>> For instance, add a source file only for WIN32 systems should be easy in
>> said declarative format.
>> Using a custom language (based on JSON for instance) would mean to add
>> conditionals, which comes back to making a custom language again.
>>
>>
>> To come back to my first point, I understand completely that this would
>> be a tremendous change, and the transition would be difficult to say the
>> least. But I think it would be more than worth it in the long term.
>>
>> > The moment you make CMake scriptable in more than one language, you are
>> forcing
>> > every CMake user to learn that additional language because sooner or
>> later he
>> > will step into a third-party that is using that additional language.
>>
>> What I have in mind is to deprecate the current CMake language and
>> replace it with another language. So there would be a transition period,
>> but in the end there would only be one language again, and a better one.
>>
>> If CMake transitioned to python (or Lua, or whatever) newcomers to CMake
>> would not need learn a new language (or at least learn one that has many
>> resources, and can be used for other purposes).
>> Old-timers would have to learn a new language (or maybe not, most
>> programmers I know have played a bit with python to automate simple tasks),
>> but this would be easier than learning CMake was, since there are
>> established rules and a more consistent design to it.
>>
>> Of course I'm not saying this should happen overnight, nor am I saying
>> this *must* happen, but I think discussing it can only be beneficial.
>>
>> I've seen lots of people wonder how to make their CMake scripts more
>> efficient, and currently this is a very difficult thing to do, since there
>> is no profiling possible.
>> And who can say they never spent way too much time trying to understand
>> why a variable was not correctly initialized ? When the configure step
>> takes about 30 seconds, and all you can do is use MESSAGE() to find what
>> happens, this is no walk in the park. A real debugger would do a world of
>> good to CMake.
>> I have seen some hardly understandable CMake code, and only thanks to the
>> git history was I able to understand that the person who wrote the script
>> co

Re: [cmake-developers] CMake alternative language (was: Using CMake as a library from Python)

[Charles Huet]

 pure
functional language up to the level of my POC and we could use it as a more
cconcrete discussion support.

Best

Le lun. 11 janv. 2016 à 10:54, Petr Kmoch <petr.km...@gmail.com> a écrit :

> Hi all.
>
> I'd like to voice my opinion as a somewhat advanced CMake user here.
>
> For me, one of the strongest points of CMake is the fact that its project
> specification is procedural rather than declarative. In my current job, for
> example, we have a significant framework built in CMake which handles a lot
> of unique setups we have (which were largely inherited from a previous
> inhouse buildsystem). Yes, the end result of our framework is that the
> CMakeLists consist mostly of declarative commands from our framework, but
> the implementation of these commands is heavily procedural. I am convinced
> that if CMake didn't give us the procedural power required to make this
> work, we couldn't have adopted it. (I had previously tried emulating bits
> of this setup in a more declarative system and failed miserably).
>
> Of course (having written much of this framework I'm talking about above),
> I know all too well that a better front-end language would do a world of
> good for CMake. I also understand that taking a more declarative approach
> could help that, and I'm not opposed to such a change in principle.
> However, please take care not to throw the baby out with the bathwater and
> nerf the expressiveness of what can be done with CMake (in a procedural
> way).
>
> If I understand Brad's suggestion correctly, it would amount to a
> (possibly empty) procedural step being used to generate a declarative
> description of the buildsystem. This would work well in our scenario, I
> believe, as long as that procedural step could be sufficiently modularised
> on the client side.
>
> I fully support introducing an alternative input language to CMake and
> taking all steps necessary for this to happen, but please do this in a way
> which will not restrict what CMake is capable of doing.
>
> Petr
>
> On Fri, Jan 8, 2016 at 5:30 PM, Brad King <brad.k...@kitware.com> wrote:
>
>> Hi Charles,
>>
>> Thanks for your efforts in exploring this topic.  CMake's current language
>> grew incrementally out of something that was not originally intended as a
>> programming language.  The cmState refactoring Stephen Kelly has started
>> is a huge step toward enabling alternative languages, but there is a long
>> way to go.
>>
>> A few general thoughts:
>>
>> * One rule we have for CMake is to never expose any public SDK of the C++
>>   implementation structures.  We want to be able to rewrite them
>> arbitrarily
>>   at any time.  Therefore any solution that needs to access the C++
>>   structures must be integrated into CMake upstream and expose
>> functionality
>>   only through other languages or file formats.
>>
>> * The cmState infrastructure builds on a "snapshot" design with a goal of
>>   being able to "fork" configuration/generation temporarily and then
>> revert
>>   back, and to be able to re-start configuration from the middle.  These
>>   goals may be incompatible with any language whose implementation we do
>>   not fully control unless it is allowed to execute only in isolated and
>>   independent snippets.  These are not hard goals, but it is a trade-off
>>   to keep in mind.  Stephen may be able to elaborate more on the snapshot
>>   approach if needed.
>>
>> * A problem with the current design is that the entire configuration
>> process
>>   is logically serial making parallel evaluation hard or impossible.  In
>>   many cases each add_subdirectory can be processed independently, but
>> this
>>   will require semantic changes to allow.
>>
>> On 01/04/2016 02:41 AM, Charles Huet wrote:
>> > I'm trying to be as declarative as possible, because really like how
>> readable
>> > simple QML programs are, and I think it would be perfect for a
>> buildsystem.
>>
>> Ideally most of the specification (sources, libraries, executables, etc.)
>> should be in a pure format that can be evaluated without side effects
>> (e.g.
>> declarative or functional).  This rules out both Python and Lua, but the
>> specification format does not have to be the main entry point.  There
>> could
>> be some imperative configuration step that does system introspection and
>> then loads the pure specification and evaluates it as needed for the
>> specific
>> environment.
>>
>> If we're going to go through the effort to provide an alternative input
>> format,
>> I think we shoul

Re: [cmake-developers] CMake alternative language

[Charles Huet]

I think this problem already exists.
When "projects build elaborate macro/function systems in the CMake
language", you have to understand the build system program. And in my
experience, those systems are widely different from one another.

I have seen a system where you defined lots of variables, then just
imported a file which used those to perform the add_library and such in the
X-Macro pattern (
https://en.wikibooks.org/wiki/C_Programming/Preprocessor#X-Macros).

I think having a language that allows for more terse expressions would be a
net gain.


Also, I find ironic to preach for a simple language when CMake is mainly
targetted at C++.
C++ programmers are used to having *lots* of very powerful tools at their
disposal (template meta-programming, operator overloading, macros, etc) and
having to choose which one to use for the job.

If you do not trust your team mates to keep the buildsystem simple and to
the point even if they had lots of stuff at their disposal, how can you
trust them with the C++ program ?


Le jeu. 14 janv. 2016 à 09:04, CHEVRIER, Marc  a
écrit :

> I fully agree with Alexander.
> It seems a very bad idea to use a general purpose script language to
> manage CMake stuff.
>
> For sure, CMake language is a bit “curious” :) and changing for a more
> user friendly one is a good idea but, in my opinion, only dedicated
> languages (I.e. Specialised for the problem to solve) must be considered.
>
> May be a good approach is to enhance current language for more flexibility
> (for example adding capability for functions to return value, etc…)
>
> Marc
>
>
>
>
>
> On 13/01/16 22:21, "cmake-developers on behalf of Alexander Neundorf" <
> cmake-developers-boun...@cmake.org on behalf of neund...@kde.org> wrote:
>
> >On Wednesday, January 13, 2016 10:59:39 yann suisini wrote:
> >> Hi,
> >>
> >> I'm a new user of CMake, but I just want to express my newcomer point of
> >> view.
> >> Honestly , I can feel the power of CMAKE, but it's a real pain to learn
> ...
> >> I'm using CMAKE for an embedded platform with a non GNU compiler , ant
> at
> >> the end the CMAKE description is longer than the one I built directly in
> >> Ninja.
> >> I had to write a python script to parse my eclipse project xml to
> create a
> >> list of sources files usable by CMAKE.
> >> The first thing I thought was: why this is not a part of cmake ? And the
> >> second thing was : why not using the scripting power of an existing
> >> language like Python(or other one) and add CMAKE as a framework /
> library ?
> >
> >My personal opinion: if the full power of python would be available, the
> build
> >scripts would quickly turn into real programs, because programmers would
> be
> >tempted to do so if all of python would be available. Then developers
> would
> >have to understand two programs: the program itself, and the "build
> program".
> >
> >I'm not saying that the cmake language is beautiful, but it helps to keep
> >cmake scripts somewhat simple, and not evolve into a second program
> >additionally to the actual program which is being built.
> >
> >Alex
> >
> >--
> >
> >Powered by www.kitware.com
> >
> >Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
> >
> >Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
> >
> >CMake Support: http://cmake.org/cmake/help/support.html
> >CMake Consulting: http://cmake.org/cmake/help/consulting.html
> >CMake Training Courses: http://cmake.org/cmake/help/training.html
> >
> >Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
> >
> >Follow this link to subscribe/unsubscribe:
> >http://public.kitware.com/mailman/listinfo/cmake-developers
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow 

Re: [cmake-developers] CMake alternative language

[Charles Huet]

As long as CMake embeds everything that is required, I don't see the
additional pain (since this is what it already does with the CMake
language).


Le jeu. 14 janv. 2016 à 13:35, Jean-Michaël Celerier <
jeanmichael.celer...@gmail.com> a écrit :

>
> On Wed, Jan 13, 2016 at 10:21 PM, Alexander Neundorf 
> wrote:
>
>>
>> My personal opinion: if the full power of python would be available, the
>> build
>> scripts would quickly turn into real programs, because programmers would
>> be
>> tempted to do so if all of python would be available. Then developers
>> would
>> have to understand two programs: the program itself, and the "build
>> program".
>
>
> The problem is when you have to do a real program for your build system
> anyways (which occurs one day or another for any large enough project I
> guess).
> Currently it's a real pain (but it'd be even more painful to have to ship
> Python / Bash / $script_language as part of your build system).
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CMake alternative language

[Charles Huet]

Thanks for the insight !

I started going towards python because I am more familiar with it, but Lua
seems to be a better fit for CMake now that I think about the constraints
you listed.

The main point I am getting from your mail is that Kitware may not want to
go down this route, even if provided with a working prototype, with the
ability of slow migration by using both CMake script and Lua.

I understand that some users are against such a change (as pointed out in
this very thread) but I'd be willing to contribute time and effort towards
it, if given the assurance it would not be in vain.

>I try really hard to keep my CMake really simple. But unfortunately,
it always gets complicated. Just the fact that platforms and
environments are so different always impacts my CMake scripts. But I’m
very glad that CMake seems to get the job done. Lua would help me
avoid fighting the CMake language, and that would be very welcome. But
I don’t expect the messiness to ever truly go away…that is the
byproduct of systems/environments being so different and ever
changing.

This perfectly sums up what I feel :)

Le jeu. 14 janv. 2016 à 14:46, Eric Wing  a écrit :

> I am hesitant to add more fuel to the fire for this discussion because
> it has been discussed many times before through the years and my sense
> is that this isn’t what Brad is really interested in pursuing. I
> encourage you search the mailing list history.
>
> I’ll try to summarize (off the top of my head) about why Lua (as
> opposed to other languages).
>
> - Lua above all other languages was recommended by me and others
> because it satisfied all the constraints:
> - It is tiny and can be included with the CMake source and doesn’t
> impact the size of CMake in any significant way
> - Being 100% ANSI C allows it to go everywhere CMake already goes
> and
> satisfies the requirement that only a C++ compiler is needed to
> bootstrap it.
> - Lua is a language designed to be embedded, for this very kind of
> purpose.
> - Lua is sandboxed by default and has a minimal (and optional)
> standard library.
> - This means users won’t be encouraged to call all sorts of
> non-portable library calls like with some other languages. CMake can
> control what APIs are exposed addressing those concerns that a full
> programming language may allow too much variance and craziness in the
> community.
> - Lua’s liberal license is compatible with CMake’s liberal license.
>
> - Lua is one of the cleanest, simplest mainstream languages out there
> and is incredibly easy to learn, yet still supporting a wide range of
> higher-level constructs (functions as first class values, closures,
> proper tail recursion). Documentation and tutorials are readily
> available.
> - Lua’s core data structure is the table, which is a extremely similar
> to JSON (which it happens to also predate), which makes lists and
> declarative data style very easy and natural to do.
> - Lua has proven itself in the video game industry in that C/C++
> developers are comfortable dealing with it (not a syntax shell shock
> to them, say like LISP, or quite frankly CMake)
> - Lua being a full, well designed programming language allows for ways
> to write things which is currently really hard in CMake’s language.
> Additionally, adding features to CMake’s language is hard in that you
> have to think about all the potential side-effects, edge cases, and
> inconsistencies. Lua takes this burden away and how to use everything
> is well known/documented.
>
> - To prove to us that CMake was already too far down the road and
> retrofitting Lua into CMake was not practical, Dr. Ken Martin had a
> CMake/Lua proof of concept working in like 13 hours. (Oops.)
> - But it was decided that Kitware didn’t want to go down this road.
>
> - We attempted to build off Ken Martin’s experiment to keep it alive,
> but that project essentially failed.
> - Lack of time, resources, expertise, etc, and this was back in the
> CVS days so keeping up with the mainline was hell since merge tracking
> is pretty much non-existent.
> - There was one interesting thing implemented showing that both
> existing CMake script and Lua script could co-exist. And I think there
> was even a proof a concept that different language scripts could call
> one another.
>
>
>
> So just on the constraints/requirements issue, I think we convinced
> Brad, that *if* anything should be chosen, it should be Lua, which is
> why Lua is still in his head. And these constraints are still in play.
> I am constantly fighting dependency hell on platforms. It is nice that
> CMake has no external dependencies except a C++ compiler and can
> bootstrap itself. I recently did a bunch of work on a Raspberry Pi and
> I needed to build CMake; no problem. I just tried to build Swift on
> the Pi. I got hit by a bug where the build process pulls in the wrong
> version of Python (2 vs 3) if 

[cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

Hi,

I have a folder in my build that is added as such: "add_subdirectory(folder
EXCLUDE_FROM_ALL)"

When using Unix Makefiles (or MSVC) this causes the subfolder to contain a
new Makefile, allowing me to build easily the whole subdir.

I cannot find such a facility with Ninja, which is very annoying (since the
folder I add this way are demos, there are quite a lot of targets to build).

Am I missing something or is this a feature I could add ?

Best
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

Hi,

I was looking into this yesterday, and it seems non-trivial as far as code
modifications go to make this happen.

But I wonder if this would even be the best way to go.
Having multiple ninja files would mean (as far as I understand) duplicating
lots of values in the sub ninja files.

I think it would be easier to just add a new target for each folder added
with 'EXCLUDE_FROM_ALL' that groups all the targets added within.
Maybe even adding such a target for each 'project', which regroups all the
targets of a project would make sense, as these are grouped as 'projects'
in MSVC, whereas a folder added with 'EXCLUDE_FROM_ALL' is added as a
'solution'.

This would make ninja easy to use in a fashion relatively similar to MSVC.

To avoid name collisions these could be suffixed with something like
'_dir', '_project' or such.

Do you think this would be a good enough solution ?

Le mer. 10 févr. 2016 à 19:47, Brad King <brad.k...@kitware.com> a écrit :

> On 02/10/2016 10:57 AM, Charles Huet wrote:
> > When using Unix Makefiles (or MSVC) this causes the subfolder to contain
> > a new Makefile, allowing me to build easily the whole subdir.
> >
> > I cannot find such a facility with Ninja
>
> The Makefile generator naturally lays makefiles out with launch points
> in each directory.  The Xcode generator and Visual Studio generators
> generate projects that can be loaded in the respective IDEs for each
> source directory with a project() command in its CMakeLists.txt file:
>
>
> https://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmGlobalXCodeGenerator.cxx;hb=v3.4.3#l407
>
> https://cmake.org/gitweb?p=cmake.git;a=blob;f=Source/cmGlobalVisualStudio7Generator.cxx;hb=v3.4.3#l383
>
> They contain the relevant subsets of the build system needed for their
> subdirectories.  Currently the Ninja generator does not have this
> feature.  It may be possible to add but would take some investigation
> into how intrusive the changes would be.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CTest custom HTML

[Charles Huet]

Hi,

this is very interesting, but it does not fully solve the problem of not
being able to have some custom HTML sent to CDash directly.

I perused through the code to try and find a solution, and I think I have
an idea.
>From what I understand, the 'DartMeasurement' tokens are replaced with some
logic into  and such.

If this were to happen in an XSLT file, that could be specified to CTest,
then not only would the current behavior for vanilla CTest/CDash remain the
same, it would be easier to maintain and make evolve as it would not
require recompilation.
On top of that, any custom behavior like ours is very easy to support, just
specify a customn XSLT sheet, and have CTest use this one instead of the
default.

Using an XSLT2  character-map would allow for the current behavior of
replacing characters like '>', '<' and other HTML-breaking things quite
easily.

Now the big difficulty in this would be to integrate XSLT2 capabilities
into CMake.
As far as I understand, CMake tries very hard to be as cross-platform as
possible in a single package, which means no hard dependencies, and any
third-party lib gets integrated into the CMake source tree (such as
cmbzip2, cmcurl, etc).

I only know of libxslt, and have no idea if it can be integrated in this
way, or how much work that would be.
Maybe there are better, lighter alternatives out there, but I am not
knowledgable in xslt libraries.

Do you think this coule be viable ?


Le mer. 9 mars 2016 à 14:36, Brad King <brad.k...@kitware.com> a écrit :

> On 03/09/2016 04:13 AM, Charles Huet wrote:
> > We used to use this, by creating a composite image containing the
> reference,
> > generated and diff, but in some cases comparing accurately was tedious.
> > Having the 3 images separately allows us to open each in a tab and
> quickly
> > switch from one to the other, making human comparison much easier.
> >
> > Also, we have added some HTML buttons and tables, which we would lose by
> using this.
>
> This work seems more suited for development on the CDash side to influence
> how it displays the images.
>
> For VTK testing we have an interactive slider to help compare the images:
>
>  https://open.cdash.org/testDetails.php?test=422454265=4246600
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CTest custom HTML

[Charles Huet]

Yeah, makes sense.

Thanks so much for your time

Le lun. 14 mars 2016 à 15:58, Brad King <brad.k...@kitware.com> a écrit :

> On 03/14/2016 05:13 AM, Charles Huet wrote:
> > If the CTest parser parsed the output as XML and not simple text, it
> should be
> > able to recognize CDATA from actual markup.
> [snip]
> > If you don't think this is doable (or you don't want to integrate this
> kind
> > of behavior into CTest) we'll customize CDash and I'll stop badgering
> you about this.
> >
> > But I'd rather do this as a last resort.
>
> I'm sorry but I don't want to introduce any special interpretation of test
> output beyond the hooks that are already present.  In principle the test
> output is just plain text and the fact that it could be interpreted as
> XML or JSON or some other markup is not CTest's business.  It is encoded
> in Test.xml for submission to CDash or some other similar system with
> all escaping needed to communicate the real test output to that system.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

Hi,

I modified my patch a bit to use C++98 only, and try to stick to the coding
conventions.

This now adds a target per folder, which contains all the targets in the
folder and all its children folders.

These targets are prefixed with "path/" in order to set them apart of
normal CMake targets.

Please tell me if more modifications are required.
Best.

Le ven. 11 mars 2016 à 16:58, Brad King <brad.k...@kitware.com> a écrit :

> On 03/11/2016 10:48 AM, Charles Huet wrote:
> > Here is a prototype that adds a target for each folder the
> > has the EXCLUDE_FROM_ALL property set.
>
> Is there any reason to limit this to EXCLUDE_FROM_ALL?  Why not
> have a target for every directory with a CMakeLists.txt file?
>
> > I am not satisfied with the way I exclude some build targets,
> > but I could not find the correct way to do it (Install target and such).
>
> See cmGlobalUnixMakefileGenerator3::WriteDirectoryRule2 and
> its call sites for how the Makefile generator decides which
> targets are part of "all" for each directory.  These targets
> in build.ninja should be equivalent to "cd dir; make" in the
> Makefile generator.
>
> > I used C++11, but I don't know if this is OK in CMake, I'll refactor if
> not.
>
> Please use C++98.
>
> Thanks!
> -Brad
>
>
From f3417b48ed8b676b10c06ede83e38fce6703f29a Mon Sep 17 00:00:00 2001
From: Charles Huet <charles.h...@gmail.com>
Date: Fri, 11 Mar 2016 16:26:29 +0100
Subject: [PATCH] Added a target for each EXCLUDED_FROM_ALL folder that holds
 all the targets in said folder

---
 Source/cmGlobalNinjaGenerator.cxx | 56 +++
 1 file changed, 56 insertions(+)

diff --git a/Source/cmGlobalNinjaGenerator.cxx b/Source/cmGlobalNinjaGenerator.cxx
index 0f06e43..050bc0e 100644
--- a/Source/cmGlobalNinjaGenerator.cxx
+++ b/Source/cmGlobalNinjaGenerator.cxx
@@ -1040,6 +1040,62 @@ void cmGlobalNinjaGenerator::WriteTargetAliases(std::ostream& os)
   cmNinjaDeps(1, i->first),
   deps);
   }
+
+  cmGlobalNinjaGenerator::WriteDivider(os);
+  os << "# Folder targets.\n\n";
+
+  std::map<std::string, cmNinjaDeps > targetsPerFolder;
+
+  for ( std::vector::const_iterator generatorIt = this->LocalGenerators.begin();
+generatorIt != this->LocalGenerators.end();
+++generatorIt)
+{
+const cmLocalGenerator* localGenerator = *generatorIt;
+const std::string currentSourceFolder( localGenerator->GetStateSnapshot().GetDirectory().GetCurrentSource() );
+targetsPerFolder[currentSourceFolder] = cmNinjaDeps();
+for ( std::vector<cmGeneratorTarget*>::const_iterator targetIt = localGenerator->GetGeneratorTargets().begin();
+  targetIt != localGenerator->GetGeneratorTargets().end();
+  ++targetIt)
+  {
+  const cmGeneratorTarget* generatorTarget = *targetIt;
+
+  const int type = generatorTarget->GetType();
+  if((type == cmState::EXECUTABLE) ||
+ (type == cmState::STATIC_LIBRARY) ||
+ (type == cmState::SHARED_LIBRARY) ||
+ (type == cmState::MODULE_LIBRARY) ||
+ (type == cmState::OBJECT_LIBRARY) ||
+ (type == cmState::UTILITY))
+{
+
+// insert the current target in every folder whose name contains the current target's folder
+for ( std::map<std::string, cmNinjaDeps >::iterator it = targetsPerFolder.begin(); it != targetsPerFolder.end(); ++it )
+  {
+  if( currentSourceFolder.find( it->first ) != std::string::npos )
+{
+it->second.push_back( generatorTarget->GetName() );
+}
+  }
+}
+  }
+}
+
+  for ( std::map<std::string, cmNinjaDeps >::const_iterator it = targetsPerFolder.begin(); it != targetsPerFolder.end(); ++it )
+{
+if ( it->second.empty() )
+  continue;
+
+cmGlobalNinjaGenerator::WriteDivider( os );
+
+cmLocalGenerator* firstLocalGenerator = this->LocalGenerators[0];
+const std::string rootSourceDir = firstLocalGenerator->GetSourceDirectory();
+const std::string folderRelativeToSource = "path/" +  std::string(it->first).replace( 0, rootSourceDir.size() + 1, "" );
+const std::string comment = "Folder: " + std::string(it->first);
+cmNinjaDeps output(1);
+output.push_back( folderRelativeToSource );
+
+this->WritePhonyBuild(os, comment, output, it->second);
+}
 }
 
 void cmGlobalNinjaGenerator::WriteUnknownExplicitDependencies(std::ostream& os)
-- 
1.8.3.1

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: htt

Re: [cmake-developers] CTest custom HTML

[Charles Huet]

>You could use DartMeasurementFile to upload arbitrary
files with the CTest submission, and then teach CDash what to do with
them.

Hmm, this sounds to me like too much custom logic would be in CDash, which
would never make it upstream, and we have bad experiences maintaining this
kind of patch sets.

>I'd prefer not to add yet another dependency (on xslt).
Yeah, I guessed this would be your answer, it makes sense.

I have had another, simpler (?) idea that would fit everybody's
requirements.
If the CTest parser parsed the output as XML and not simple text, it should
be able to recognize CDATA from actual markup.
Thus replacing the special chars could be done when reading, and the
cmCtestTestHandler::GenerateDartOutput() function could be adapted
easily(?) to write the read XML.

I'm not sure how feasible this would be, but this sounds like the best way
to integrate custom XML in a CTest report.

If you don't think this is doable (or you don't want to integrate this kind
of behavior into CTest) we'll customize CDash and I'll stop badgering you
about this.

But I'd rather do this as a last resort.

Thanks


Le jeu. 10 mars 2016 à 15:29, Brad King <brad.k...@kitware.com> a écrit :

> On 03/10/2016 03:29 AM, Charles Huet wrote:
> > Now the big difficulty in this would be to integrate XSLT2 capabilities
> into CMake.
> [snip]
> > Do you think this coule be viable ?
>
> I'd prefer not to add yet another dependency (on xslt).  CTest defines
> nothing about the format of the test output, besides DartMeasurement hooks.
> It isn't guaranteed to even be displayed on a web page where having
> custom HTML even makes sense.  This really should be done on the
> CDash side.  You could use DartMeasurementFile to upload arbitrary
> files with the CTest submission, and then teach CDash what to do with
> them.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

Here is a prototype that adds a target for each folder the has the
EXCLUDE_FROM_ALL property set.

Please review it and I'll get it in acceptable shape.

I used the folder name as target for now, this is not final but it was easy
for prototyping.
I am not satisfied with the way I exclude some build targets, but I could
not find the correct way to do it (Install target and such).

I used C++11, but I don't know if this is OK in CMake, I'll refactor if not.

Best

Le jeu. 10 mars 2016 à 15:28, Brad King <brad.k...@kitware.com> a écrit :

> On 03/10/2016 02:58 AM, Charles Huet wrote:
> > Maybe even adding such a target for each 'project', which regroups all
> the
> > targets of a project would make sense, as these are grouped as 'projects'
> [snip]
> > To avoid name collisions these could be suffixed with something like
> '_dir',
> > '_project' or such.
> >
> > Do you think this would be a good enough solution ?
>
> I think such targets makes sense and does not disallow full subproject
> generation in the future.  To avoid name collisions we should use a
> prefix or suffix with a "/" in its name since that is not allowed by
> CMake for the actual target names.
>
> Thanks,
> -Brad
>
>
From e95c65c80501444358c50d7c4d44ca6759ea9c9f Mon Sep 17 00:00:00 2001
From: Charles Huet <charles.h...@gmail.com>
Date: Fri, 11 Mar 2016 16:26:29 +0100
Subject: [PATCH] Added a target for each EXCLUDED_FROM_ALL folder that holds
 all the targets in said folder

---
 Source/cmGlobalNinjaGenerator.cxx | 59 +++
 1 file changed, 59 insertions(+)

diff --git a/Source/cmGlobalNinjaGenerator.cxx b/Source/cmGlobalNinjaGenerator.cxx
index 0f06e43..76aed95 100644
--- a/Source/cmGlobalNinjaGenerator.cxx
+++ b/Source/cmGlobalNinjaGenerator.cxx
@@ -1040,6 +1040,65 @@ void cmGlobalNinjaGenerator::WriteTargetAliases(std::ostream& os)
   cmNinjaDeps(1, i->first),
   deps);
   }
+
+  cmGlobalNinjaGenerator::WriteDivider(os);
+  os << "# Folder targets.\n\n";
+
+  std::map<std::string, cmNinjaDeps > targetsPerFolder;
+  std::vector excludedSourceFolders;
+
+  auto isExcludedFromAll = []( const cmLocalGenerator* localGenerator ) -> std::string {
+for ( const std::string& folder : excludedSourceFolders )
+{
+  const std::string currentSourceDir( localGenerator->GetStateSnapshot().GetDirectory().GetCurrentSource() );
+  if ( currentSourceDir.find( folder ) != std::string::npos )
+  {
+return folder;
+  }
+}
+return std::string();
+  };
+
+  for ( const cmLocalGenerator* localGenerator : this->LocalGenerators )
+  {
+if ( localGenerator->GetStateSnapshot().GetDirectory().GetPropertyAsBool( "EXCLUDE_FROM_ALL" ) )
+{
+  excludedSourceFolders.push_back( localGenerator->GetStateSnapshot().GetDirectory().GetCurrentSource() );
+}
+
+const std::string excludedSourceFolder = isExcludedFromAll( localGenerator );
+if ( !excludedSourceFolder.empty() )
+{
+  for ( const cmGeneratorTarget* generatorTarget : localGenerator->GetGeneratorTargets() )
+  {
+if ( generatorTarget->GetName() != GetTestTargetName() &&
+ generatorTarget->GetName() != GetInstallLocalTargetName() &&
+ generatorTarget->GetName() != GetInstallStripTargetName() &&
+ generatorTarget->GetName() != GetInstallTargetName() &&
+ generatorTarget->GetName() != GetEditCacheTargetName() &&
+ generatorTarget->GetName() != GetRebuildCacheTargetName() &&
+ generatorTarget->GetName() != GetRebuildCacheTargetName() &&
+ generatorTarget->GetName().find( "list_install_components" ) == std::string::npos &&
+ generatorTarget->GetName().find( "_automoc" ) == std::string::npos )
+{
+  targetsPerFolder[excludedSourceFolder].push_back( generatorTarget->GetName() );
+}
+  }
+}
+  }
+
+  for ( auto it = targetsPerFolder.begin(); it != targetsPerFolder.end(); ++it )
+  {
+if ( it->second.empty() )
+  continue;
+
+cmGlobalNinjaGenerator::WriteDivider( os );
+
+cmNinjaDeps output(1);
+output.push_back( it->first );
+
+this->WritePhonyBuild(os, "Folder: " + std::string(it->first), output, it->second);
+  }
 }
 
 void cmGlobalNinjaGenerator::WriteUnknownExplicitDependencies(std::ostream& os)
-- 
1.8.3.1

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake 

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

Awesome, thanks!

I'll check monday and report, but this looks like it'll work the same as
the patch I sent last, which I've been using the past few days.
Looking forward to have this feature in CMake !

Le ven. 18 mars 2016 à 16:03, Brad King <brad.k...@kitware.com> a écrit :

> On 03/17/2016 11:23 AM, Charles Huet wrote:
> > I added a utility function 'ConvertToNinjaFolderRule' that
> > does the same thing as ConvertToNinjaPath, but makes relative
> > to the source dir, and adds the '/all' suffix.
>
> Great.  I've applied the patch with a few style changes and
> logic fixes.  I also added documentation and a test:
>
>  Ninja: Add `$subdir/all` targets
>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=ca575fe9
>
>  Ninja: Add test for `$subdir/all` targets
>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=86c851e6
>
>  Help: Add notes for topic 'ninja-directory-targets'
>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=133aed8f
>
> Please try out that version to make sure it still works for you.
>
> Thanks,
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

> This is per-target and is different than add_subdirectory
EXCLUDE_FROM_ALL.
We should still honor this.

Ok, done.

> The rules we're generating here could work like that too.  Each one
depends on all the targets defined in its own directory and on the
directory-level target for subdirectories.  The goal is to match the
Makefile generator behavior for target selection in each directory.
I don't think matching subdirectory name prefixes is needed for that.

Ah, yes, this makes a lot of sense. Now I feel a bit dumb for not going
this way in the first place.

> I think the names can instead use a "/all" suffix e.g. "sub/dir/all".
This would be consistent with the "ninja all" one can now do at the
top.

Sure, done.

If you have other ideas on how to improve this patch, I'll be happy to
implement them.
I've been using Cmake 3.5 with this added feature and I must say it's
pretty sweet for my use case.

Best

Le mar. 15 mars 2016 à 18:58, Brad King <brad.k...@kitware.com> a écrit :

> On 03/14/2016 05:04 AM, Charles Huet wrote:
> > I modified my patch a bit to use C++98 only, and try to stick
> > to the coding conventions.
>
> Thanks.  While reviewing the logic I realized my suggestion to include
> all targets was not quite consistent with my observation that this matches
> the Makefile generator "cd dir; make" behavior.  The latter still excludes
> targets that have the EXCLUDE_FROM_ALL target property, as seen by this
> code in cmGlobalUnixMakefileGenerator3::WriteDirectoryRule2:
>
> >  if((!check_all || !gtarget->GetPropertyAsBool("EXCLUDE_FROM_ALL")) &&
>
> This is per-target and is different than add_subdirectory EXCLUDE_FROM_ALL.
> We should still honor this.  Such targets are meant to be built only upon
> explicit request.  Also later in WriteDirectoryRule2 is this section:
>
> > // The directory-level rule should depend on the directory-level
> > // rules of the subdirectories.
>
> The rules we're generating here could work like that too.  Each one
> depends on all the targets defined in its own directory and on the
> directory-level target for subdirectories.  The goal is to match the
> Makefile generator behavior for target selection in each directory.
> I don't think matching subdirectory name prefixes is needed for that.
>
> > These targets are prefixed with "path/" in order to set them apart of
> > normal CMake targets.
>
> I think the names can instead use a "/all" suffix e.g. "sub/dir/all".
> This would be consistent with the "ninja all" one can now do at the
> top.
>
> Thanks,
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

Yes, sorry about that.

Le mer. 16 mars 2016 à 15:46, Brad King <brad.k...@kitware.com> a écrit :

> On 03/16/2016 04:59 AM, Charles Huet wrote:
> > Sure, done.
> >
> > If you have other ideas on how to improve this patch, I'll be happy to
> > implement them.
>
> Great.  Did you mean to attach a revised patch?
>
> Thanks,
> -Brad
>
>
From 25308102899bdcab9693da736184a4dc38001b4b Mon Sep 17 00:00:00 2001
From: Charles Huet <charles.h...@gmail.com>
Date: Fri, 11 Mar 2016 16:26:29 +0100
Subject: [PATCH] Added a target for each EXCLUDED_FROM_ALL folder that holds
 all the targets in said folder

---
 Source/cmGlobalNinjaGenerator.cxx | 69 +++
 Source/cmGlobalNinjaGenerator.h   |  1 +
 2 files changed, 70 insertions(+)

diff --git a/Source/cmGlobalNinjaGenerator.cxx b/Source/cmGlobalNinjaGenerator.cxx
index 0f06e43..0fa76a0 100644
--- a/Source/cmGlobalNinjaGenerator.cxx
+++ b/Source/cmGlobalNinjaGenerator.cxx
@@ -577,6 +577,7 @@ void cmGlobalNinjaGenerator::Generate()
 
   this->WriteAssumedSourceDependencies();
   this->WriteTargetAliases(*this->BuildFileStream);
+  this->WriteFolderTargets(*this->BuildFileStream);
   this->WriteUnknownExplicitDependencies(*this->BuildFileStream);
   this->WriteBuiltinTargets(*this->BuildFileStream);
 
@@ -1042,6 +1043,74 @@ void cmGlobalNinjaGenerator::WriteTargetAliases(std::ostream& os)
   }
 }
 
+void cmGlobalNinjaGenerator::WriteFolderTargets(std::ostream& os)
+{
+  cmGlobalNinjaGenerator::WriteDivider(os);
+  os << "# Folder targets.\n\n";
+
+  cmLocalGenerator* firstLocalGenerator = this->LocalGenerators[0];
+  const std::string rootSourceDir = firstLocalGenerator->GetSourceDirectory();
+
+  std::map<std::string, cmNinjaDeps > targetsPerFolder;
+
+  for ( std::vector::const_iterator generatorIt = this->LocalGenerators.begin();
+generatorIt != this->LocalGenerators.end();
+++generatorIt)
+{
+const cmLocalGenerator* localGenerator = *generatorIt;
+const std::string currentSourceFolder(localGenerator->GetStateSnapshot().GetDirectory().GetCurrentSource());
+targetsPerFolder[currentSourceFolder] = cmNinjaDeps();
+for ( std::vector<cmGeneratorTarget*>::const_iterator targetIt = localGenerator->GetGeneratorTargets().begin();
+  targetIt != localGenerator->GetGeneratorTargets().end();
+  ++targetIt)
+  {
+  const cmGeneratorTarget* generatorTarget = *targetIt;
+
+  const int type = generatorTarget->GetType();
+  if((type == cmState::EXECUTABLE) ||
+ (type == cmState::STATIC_LIBRARY) ||
+ (type == cmState::SHARED_LIBRARY) ||
+ (type == cmState::MODULE_LIBRARY) ||
+ (type == cmState::OBJECT_LIBRARY) ||
+ (type == cmState::UTILITY) &&
+ (!generatorTarget->GetPropertyAsBool("EXCLUDE_FROM_ALL"))
+ )
+{
+targetsPerFolder[currentSourceFolder].push_back(generatorTarget->GetName());
+}
+  }
+
+  // The directory-level rule should depend on the directory-level
+  // rules of the subdirectories.
+  std::vector children
+  = localGenerator->GetStateSnapshot().GetChildren();
+  for(std::vector::const_iterator
+stateIt = children.begin(); stateIt != children.end(); ++stateIt)
+{
+std::string currentSourceDir(stateIt->GetDirectory().GetCurrentSource());
+std::string subdir = currentSourceDir.substr(rootSourceDir.length() + 1, currentSourceDir.length()) + "/all";
+targetsPerFolder[currentSourceFolder].push_back(subdir);
+}
+}
+
+  for ( std::map<std::string, cmNinjaDeps >::const_iterator it = targetsPerFolder.begin(); it != targetsPerFolder.end(); ++it )
+{
+cmGlobalNinjaGenerator::WriteDivider( os );
+std::string currentSourceDir(it->first);
+
+//do not generate a rule for the root source dir
+if(rootSourceDir.length() >= currentSourceDir.length())
+  continue;
+
+const std::string folderRule = currentSourceDir.substr(rootSourceDir.length() + 1, currentSourceDir.length()) + "/all";
+const std::string comment = "Folder: " + std::string(it->first);
+cmNinjaDeps output(1);
+output.push_back(folderRule);
+
+this->WritePhonyBuild(os, comment, output, it->second);
+}
+}
+
 void cmGlobalNinjaGenerator::WriteUnknownExplicitDependencies(std::ostream& os)
 {
   if (!this->ComputingUnknownDependencies)
diff --git a/Source/cmGlobalNinjaGenerator.h b/Source/cmGlobalNinjaGenerator.h
index 8656590..e95a295 100644
--- a/Source/cmGlobalNinjaGenerator.h
+++ b/Source/cmGlobalNinjaGenerator.h
@@ -342,6 +342,7 @@ private:
   void WriteAssumedSourceDependencies();
 
   void WriteTargetAliases(std::ostream& os);
+  void WriteFolderTargets(std::ostream& os);
   voi

[cmake-developers] CTest custom HTML

[Charles Huet]

Hi,

to customize our test results, we developped a small test wrapper that
outputs some HTML markup (mostly useful to display images, since some of
our tests compare images with ImageMagick).
However, this does not work since CMake 3.3 because of the refactoring of
ctest to use cmXMLWriter, because cmXMLWriter is smart and wants to output
correct XML, and thus escapes everything.

While I completely understand and applaud the motivation and work behind
cmXMLWriter, it annoys me that we cannot have our custom reporting anymore.

Sadly, I have no idea how to have both the correctness provided by
cmXMLWriter and the flexibility of outputting custom HTML via the test's
stdout.

How could this be done with CMake > 3.3 ?
If it needs additional development on the CTest side I would of course try
and accomplish it, if anyone has an idea.

Thanks
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] CTest custom HTML

[Charles Huet]

Sadly this is quite not good enough.
We used to use this, by creating a composite image containing the
reference, generated and diff, but in some cases comparing accurately was
tedious.
Having the 3 images separately allows us to open each in a tab and quickly
switch from one to the other, making human comparison much easier.

Also, we have added some HTML buttons and tables, which we would lose by
using this.



Le mar. 8 mars 2016 à 19:27, Brad King <brad.k...@kitware.com> a écrit :

> On 03/08/2016 11:09 AM, Charles Huet wrote:
> > mostly useful to display images, since some of our tests compare
> > images with ImageMagick
>
> CTest/CDash already supports sending images for display.  It was
> developed for the same purpose for VTK testing.  The test output
> can contain
>
>type="image/png">/path/to/file.png
>
> CTest will load the named file and send it as part of the submission
> to CDash.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Ninja and add_subdirectory EXCLUDE_FROM_ALL

[Charles Huet]

ConvertToNinjaPath sadly makes the path relative to the binary dir, not the
source dir.

I added a utility function 'ConvertToNinjaFolderRule' that does the same
thing as ConvertToNinjaPath, but makes relative to the source dir, and adds
the '/all' suffix.

I also made the lines less than 79 chars.

Best


Le jeu. 17 mars 2016 à 15:53, Brad King <brad.k...@kitware.com> a écrit :

> On 03/16/2016 12:13 PM, Charles Huet wrote:
> >> On 03/16/2016 04:59 AM, Charles Huet wrote:
> >>> If you have other ideas on how to improve this patch, I'll be happy to
> >>> implement them.
>
> All paths that are given to WritePhonyBuild in the outputs and depends
> options should be sent through ConvertToNinjaPath.  This makes paths
> relative and formats slashes for Windows.  This should avoid the need
> to do string manipulation on the paths (e.g. substr) too.  Just be sure
> to append the "/all" before calling the conversion method.
>
> Also please keep source lines to 79 characters or less.
>
> Thanks,
> -Brad
>
>
From d204066149e098b9e33fc1004807a6d19d3f022b Mon Sep 17 00:00:00 2001
From: Charles Huet <charles.h...@gmail.com>
Date: Fri, 11 Mar 2016 16:26:29 +0100
Subject: [PATCH] Added a target for each EXCLUDED_FROM_ALL folder that holds
 all the targets in said folder

---
 Source/cmGlobalNinjaGenerator.cxx | 85 +++
 Source/cmGlobalNinjaGenerator.h   |  3 ++
 2 files changed, 88 insertions(+)

diff --git a/Source/cmGlobalNinjaGenerator.cxx b/Source/cmGlobalNinjaGenerator.cxx
index 0f06e43..49acfe3 100644
--- a/Source/cmGlobalNinjaGenerator.cxx
+++ b/Source/cmGlobalNinjaGenerator.cxx
@@ -577,6 +577,7 @@ void cmGlobalNinjaGenerator::Generate()
 
   this->WriteAssumedSourceDependencies();
   this->WriteTargetAliases(*this->BuildFileStream);
+  this->WriteFolderTargets(*this->BuildFileStream);
   this->WriteUnknownExplicitDependencies(*this->BuildFileStream);
   this->WriteBuiltinTargets(*this->BuildFileStream);
 
@@ -849,6 +850,18 @@ std::string cmGlobalNinjaGenerator::ConvertToNinjaPath(const std::string& path)
   return convPath;
 }
 
+std::string cmGlobalNinjaGenerator::ConvertToNinjaFolderRule(const std::string& path)
+{
+  cmLocalNinjaGenerator *ng =
+static_cast(this->LocalGenerators[0]);
+  std::string convPath = ng->Convert(path, cmOutputConverter::HOME);
+#ifdef _WIN32
+  cmSystemTools::ReplaceString(convPath, "/", "\\");
+#endif
+  return convPath + "/all";
+}
+
+
 void cmGlobalNinjaGenerator::AddCXXCompileCommand(
   const std::string ,
   const std::string )
@@ -1042,6 +1055,78 @@ void cmGlobalNinjaGenerator::WriteTargetAliases(std::ostream& os)
   }
 }
 
+void cmGlobalNinjaGenerator::WriteFolderTargets(std::ostream& os)
+{
+  cmGlobalNinjaGenerator::WriteDivider(os);
+  os << "# Folder targets.\n\n";
+
+  cmLocalGenerator* firstLocalGenerator = this->LocalGenerators[0];
+  const std::string rootSourceDir = firstLocalGenerator->GetSourceDirectory();
+
+  std::map<std::string, cmNinjaDeps > targetsPerFolder;
+
+  for ( std::vector::const_iterator generatorIt =
+this->LocalGenerators.begin();
+generatorIt != this->LocalGenerators.end();
+++generatorIt)
+{
+const cmLocalGenerator* localGenerator = *generatorIt;
+const std::string currentSourceFolder(
+  localGenerator->GetStateSnapshot().GetDirectory().GetCurrentSource());
+targetsPerFolder[currentSourceFolder] = cmNinjaDeps();
+for ( std::vector<cmGeneratorTarget*>::const_iterator targetIt =
+  localGenerator->GetGeneratorTargets().begin();
+  targetIt != localGenerator->GetGeneratorTargets().end();
+  ++targetIt)
+  {
+  const cmGeneratorTarget* generatorTarget = *targetIt;
+
+  const int type = generatorTarget->GetType();
+  if((type == cmState::EXECUTABLE) ||
+ (type == cmState::STATIC_LIBRARY) ||
+ (type == cmState::SHARED_LIBRARY) ||
+ (type == cmState::MODULE_LIBRARY) ||
+ (type == cmState::OBJECT_LIBRARY) ||
+ (type == cmState::UTILITY) &&
+ (!generatorTarget->GetPropertyAsBool("EXCLUDE_FROM_ALL"))
+ )
+{
+targetsPerFolder[currentSourceFolder].push_back(
+  generatorTarget->GetName());
+}
+  }
+
+  // The directory-level rule should depend on the directory-level
+  // rules of the subdirectories.
+  std::vector children
+  = localGenerator->GetStateSnapshot().GetChildren();
+  for(std::vector::const_iterator
+stateIt = children.begin(); stateIt != children.end(); ++stateIt)
+{
+targetsPerFolder[currentSourceFolder].push_back(
+  Conv

Re: [cmake-developers] Question over language bindings / daemon

[Charles Huet]

Hi Rick,

this is something I played with a bit, and while it is completely possible
to make your own little python binding, it will not be merged into CMake,
which means you will have to maintain it on your side.
Also, you will need to write the whole interface yourself, as the current
CMake lib API is not meant to be used in such a way.

While I too am more familiar and at ease with python and its ecosystem, lua
is a better fit for CMake as it can be redistribued easily (just one C file
to add to the build).
This would allow for better control of the updates, avoiding potential
conflicts with the installed version on the system and so on.

My Proof of concept can be found easily, but it will remain in this
unfinished state for ever, since when I'll get time to work on this I will
pursue the Lua angle.

Best,
Charles


Le lun. 18 juil. 2016 à 14:47, Tobias Hunger  a écrit :

> Hi Rick,
>
> On Mo, 2016-07-18 at 00:59 +0100, RW via cmake-developers wrote:
> > Question 1
> > Will the new daemon mode if it's included have all the commands
> associated
> > with a CMakeFile?
> > (In other-words could I write a python library that could just pipe
> > commands into it, instead of generating a separate CMakeFile)
>
> It's server mode now:-)
>
> No, daemon mode will provide commands to query the build graph etc., but
> it will
> not be usable to feed CMakeLists.txt-commands to cmake one by one.
>
> > Question 2
> > I had a look at the way the functions are written within the code
> > while I could extract a list of them using something like CppSharp (for
> > example to extract a list of everything that inherits from cmCommand)
> > There doesn't seem to be any metadata that describes arguments that are
> > passed into a given function. (only that each function takes a vector of
> > strings).
> > Do you know if there's any xml or associated metadata (perhaps used to
> > generate the docs) for which arguments are associated with a given
> function?
>
> This in turn is in scope for server mode (not at the start, but later):
> That
> information is a by-product of the idea of server mode helping IDEs with
> code-
> completion of CMakeLists.txt files.
>
> Best Regards,
> Tobias
>
> --
> Tobias Hunger, Senior Software Engineer | The Qt Company
> The Qt Company GmbH, Rudower Chaussee 13, D-12489 Berlin
> Geschäftsführer: Mika Pälsi, Juha Varelius, Mika Harjuaho. Sitz der
> Gesellschaft: Berlin, Registergericht: Amtsgericht Charlottenburg, HRB
> 144331 B
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [CMake] Need ideas/opinions on third party library management

[Charles Huet]

Hi,

I was looking at tools that can do this kind of things myself (however I
was more looking at pre-built binaries redistribution than at a
super-build, since our build time is already quite long).

Does Conan (https://conan.io/) not fit your bill as well ?

Best

Le dim. 14 août 2016 à 02:33, Ian Henriksen <
insertinterestingnameh...@gmail.com> a écrit :

> On Sat, Aug 13, 2016 at 6:43 PM Elizabeth A. Fischer <
> elizabeth.fisc...@columbia.edu> wrote:
>
>> I would look into Anaconda, which does work for Windows.  Its version
>> management is not as well developed as Spack, but it's more cross-platform.
>>
>> Auto-builders are just coming into their own, it's a brave new world.  I
>> expect things to be more complete in a few years.
>>
>> -- Elizabeth
>>
>>
> Yes, Anaconda comes with a package manager (conda) that is designed to
> manage binary dependencies. You can get a minimal Python install with just
> the
> package manager by installing miniconda. The package management system is
> built in Python, but it can manage C and C++ libraries as well. The
> support for
> ARM isn't thoroughly built out, so you'll be building all the packages
> there for
> yourself. The support for Windwos, OS X and Linux is pretty well
> developed. You
> can look at https://repo.continuum.io/pkgs/ to see the packages available
> by
> default. There is also support for people uploading their own "channels"
> for
> packages on anaconda.org. The conda-forge channel in particular has a lot
> of
> additional packages.
> Hopefully that's more like what you're looking for.
> Best,
> Ian Henriksen
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

Hi,

I rebased the Fastbuild generator on the latest master, and pushed my work
to fastbuild-master.

There are still some test fail, a few are due to an arror in manifest
management, where multiple binarie's build try to write to the same
manifest file at the same time (e.g. ObjectLibrary).
Some issues are clearly identified but not solved yet, such as
RunCMake.BuildDepends which fails because non-source dependencies are not
yet managed properly.
Some of these are new since the rebase on 3.7/master, either new tests
(e.g. android-related) or regressions, I will work at identiofying and
correcting those.

Do you think it is time to start the review, or should it wait until I have
100% of the tests passing ?

Here is a full list of failing tests:

93% tests passed, 28 tests failed out of 406
The following tests FAILED:
 46 - ObjectLibrary (Failed)
 52 - LinkDirectory (Failed)
 61 - Preprocess (Failed)
 62 - ExportImport (Timeout)
 66 - CompileDefinitions (Failed)
 76 - CMakeBuildTest (Failed)
 79 - Module.ExternalData (Failed)
108 - CustomCommand (Failed)
109 - CustomCommandByproducts (Failed)
113 - BuildDepends (Failed)
114 - SimpleInstall (Failed)
115 - SimpleInstall-Stage2 (Failed)
131 - ExternalProject (Failed)
135 - ExternalProjectUpdateSetup (Failed)
136 - ExternalProjectUpdate (Failed)
153 - Plugin (Timeout)
225 - IncludeDirectories (Failed)
226 - InterfaceLinkLibraries (Failed)
276 - RunCMake.CMP0060 (Failed)
278 - RunCMake.CMP0065 (Timeout)
281 - RunCMake.AndroidTestUtilities (Failed)
282 - RunCMake.BuildDepends (Failed)
284 - RunCMake.Configure (Failed)
345 - RunCMake.try_compile (Failed)
372 - RunCMake.CrosscompilingEmulator (Failed)
374 - RunCMake.AutoExportDll (Failed)
389 - CMake.ModuleNotices (Failed)
406 - CMake.CheckSourceTree (Failed)
Errors while running CTest

Thanks.

Le ven. 23 sept. 2016 à 17:21, Charles Huet <charles.h...@gmail.com> a
écrit :

> Sorry I pushed on fastbuild-rebase, as the rebase on master fails evento
> self host.
>
> I did not expect this to be merged for 3.7, as I'm not sure I'll have time
> to polish it enough for release.
>
> On Fri, Sep 23, 2016, 16:52 Brad King <brad.k...@kitware.com> wrote:
>
> On 09/23/2016 10:30 AM, Charles Huet wrote:
> > If you want to take a quick glance it should be close to what you want
>
> Did you push an update to the fastbuild-master branch?  It looks the same.
>
> FYI, I'd like to wait until after the Oct 3 branch for the 3.7 release
> before introducing changes toward a major new feature.  Therefore I
> likely won't have a chance to take a serious look at this until after
> that.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

>* Please use Utilities/Scripts/clang-format.bash for
>  code style.  This is easy to apply automatically later
>  so don't worry about it too much.
I'll get on this soon, I like to have proper style.


> * For final integration I'd like the commits to be squashed
>  and arranged in an organized way.  This is hard to do until
>  you have everything working though.

Sure. For now there are the first few commits which I can't change much to
keep proper attribuation, and a few commits on top which will be squashed
together once the dev is complete (or maybe earlier if I feel like it).

>* Please update the license notices to use the new style that
>  we switched to in `master`.  Both C++ and .cmake notices
>  changed.
Will do.

Le jeu. 6 oct. 2016 à 17:32, Brad King <brad.k...@kitware.com> a écrit :

> On 10/06/2016 08:32 AM, Charles Huet wrote:
> > Do you think it is time to start the review, or should it wait
> > until I have 100% of the tests passing ?
>
> Thanks for the update.  Generally my reviews for new features
> mostly look at documentation, tests, style, etc. rather than
> at implementation details (which can easily be fixed/changed
> later).  Therefore final review and integration should not start
> until everything is working.
>
> However, I'm happy to take quick glances at progress once in
> a while.  Here are some comments from the current version:
>
> * Please use Utilities/Scripts/clang-format.bash for
>   code style.  This is easy to apply automatically later
>   so don't worry about it too much.
>
> * For final integration I'd like the commits to be squashed
>   and arranged in an organized way.  This is hard to do until
>   you have everything working though.
>
> * Please update the license notices to use the new style that
>   we switched to in `master`.  Both C++ and .cmake notices
>   changed.
>
> Thanks,
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

Ok, so I will focus on windows support for now.

I squashed the commits correctly, to not lose the correct authors names,
and I started moving all that I could from the global generator to a target
generator, and simplified a few things along the way.

If you want to take a quick glance it should be close to what you want, but
there is still some work as I introduced a few regressions in the
refactoring.



Le jeu. 22 sept. 2016 à 21:14, Brad King <brad.k...@kitware.com> a écrit :

> On 09/22/2016 03:02 PM, Charles Huet wrote:
> > is linux/mac support mandatory also ?
> > Fastbuild does support those platforms, but the generator so far does
> > not work as well (or at all) on those platforms.
>
> I think it is okay to support only a subset of platforms at first.
> Please just make the generator error out early on a platform that is
> not supported, or simply not even compile on such hosts.
>
> Thanks,
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

Sorry I pushed on fastbuild-rebase, as the rebase on master fails evento
self host.

I did not expect this to be merged for 3.7, as I'm not sure I'll have time
to polish it enough for release.

On Fri, Sep 23, 2016, 16:52 Brad King <brad.k...@kitware.com> wrote:

> On 09/23/2016 10:30 AM, Charles Huet wrote:
> > If you want to take a quick glance it should be close to what you want
>
> Did you push an update to the fastbuild-master branch?  It looks the same.
>
> FYI, I'd like to wait until after the Oct 3 branch for the 3.7 release
> before introducing changes toward a major new feature.  Therefore I
> likely won't have a chance to take a serious look at this until after
> that.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] FastBuild Generator

[Charles Huet]

Hi,

As some of you may know, there is a Fastbuild generator in the works, whose
development started on CMake 3.2, and was pretty much abandonned for the
past 5 months.

I took the time to rebase it on master, and while there is a lot to clean
up (and some stuff works worse that it used to because the rebase was done
hastily), I'd like for a first step review to start, so that big design
issues can be caught right now, instead of waiting for 100% of the tests to
pass before re-writing most of the code.

How does this process goes ?
Should I create a MergeRequest on gitlab ? A Pull request on GitHub ?
patches sent by mail ?
So far the code lives in
https://github.com/packadal/CMake/tree/fastbuild-master

Thanks,
Charles.

FYI as of the rebase on CMake 3.6.1, the tests results are:

94% tests passed, 25 tests failed out of 400


Label Time Summary:

Label1=   0.66 sec (1 test)

Label2=   0.66 sec (1 test)


Total Test time (real) = 614.33 sec


The following tests FAILED:

 37 - MSManifest (Failed)

 46 - ObjectLibrary (Failed)

 61 - Preprocess (Failed)

 62 - ExportImport (Failed)

 70 - StagingPrefix (Failed)

 79 - Module.ExternalData (Failed)

109 - CustomCommand (Failed)

114 - BuildDepends (Failed)

115 - SimpleInstall (Failed)

116 - SimpleInstall-Stage2 (Failed)

132 - ExternalProject (Failed)

134 - ExternalProjectLocal (Failed)

135 - ExternalProjectUpdateSetup (Failed)

136 - ExternalProjectUpdate (Failed)

153 - Plugin (SEGFAULT)

156 - SubDir (Failed)

194 - CMakeCommands.target_link_libraries (Failed)

226 - InterfaceLinkLibraries (Failed)

275 - RunCMake.CMP0060 (Failed)

277 - RunCMake.CMP0065 (Failed)

280 - RunCMake.BuildDepends (Failed)

282 - RunCMake.Configure (Failed)

362 - RunCMake.ExternalProject (Failed)

369 - RunCMake.AutoExportDll (Failed)

400 - CMake.CheckSourceTree (Failed)
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

> IIUC this refers to:
> * http://fastbuild.org
> * https://github.com/fastbuild/fastbuild

Yes, exactly.

>From a quick glance the main problem I see is that it doesn't follow
the Makefile/Ninja generator model of organizing the generator code
into a per-target generator.

OK, so I need to make use of these.

>We can do high-level reviews here first.
Alright then.

>Please make every commit properly formatted (with clang-format) rather
than formatting after-the-fact.

I actually remembered about the clang-format thread just before writing the
first email, so I thought I should just quickly ut everything to format.

>If you've made substantial non-formatting changes from the original
>version please record yourself as the author and add a trailer to
>the commit message
I squashed everything into a big commit to make things easier for me, and
completely forgot about proper attribution of the work, so I'll squash all
of Josh's commits into one, and all of mine into another and make sure
those are correctly formatted.


To sum up, what I have to do now is:
* squash commits and format those for proper attribuation
* refactor to have a cmFastbuildTargetGenerator that inherits
cmCommonTargetGenerator
* get the tests to pass

I guess before it gets fully integrated into CMake 100% test pass is
mandatory, but is linux/mac support mandatory also ?
Fastbuild does support those platforms, but the generator so far does not
work as well (or at all) on those platforms.

Best,
Charles



Le jeu. 22 sept. 2016 à 19:26, Brad King <brad.k...@kitware.com> a écrit :

> On 09/22/2016 10:41 AM, Charles Huet wrote:
> > a Fastbuild generator in the works, whose development started on CMake
> 3.2,
> > and was pretty much abandonned for the past 5 months.
> >
> > I took the time to rebase it on master
>
> Thanks for reviving it.
>
> IIUC this refers to:
>
> * http://fastbuild.org
> * https://github.com/fastbuild/fastbuild
>
> > big design issues can be caught right now
>
> From a quick glance the main problem I see is that it doesn't follow
> the Makefile/Ninja generator model of organizing the generator code
> into a per-target generator.  See cmCommonTargetGenerator and its
> subclasses.  We've found that having a per-target generator is much
> easier than passing the target pointers around in "local" generators.
>
> > Should I create a MergeRequest on gitlab?
>
> That will be fine when you have something ready, preferably after
> addressing major architectural issues.  We can do high-level reviews
> here first.
>
> > https://github.com/packadal/CMake/tree/fastbuild-master
>
> Please make every commit properly formatted (with clang-format)
> rather than formatting after-the-fact.
>
> If you've made substantial non-formatting changes from the original
> version please record yourself as the author and add a trailer to
> the commit message such as
>
>   Co-Author: Josh Green <inbi...@gmail.com>
>
> or
>
>   Inspired-by: Josh Green <inbi...@gmail.com>
>
> depending on how much original work remains.
>
> Thanks,
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

Hi again.

I performed all the aforementioned actions, please check everything is OK
for you.

I have a few questions:
* the test CMake.CheckSourceTree does not work with Fastbuild, but I could
not get it to work with Ninja either, maybe I have an environment problem ?
* the test SimpleInstall requires trhe build tool to be able to run
multiple instances in the same hierarchy, which Fastbuild does not support.
Should I refactor the test to remove this requirement ? Or simply ignore
the test ?
* It seems SimpleInstall-Stage2 depends upon SimpleInstall, is this right ?

The branch I recently pushed has only a few tests failing, quite a few seem
to be related to a lack of non-source file dependencies as far as I can
tell.

Thanks.

Le jeu. 6 oct. 2016 à 19:33, Charles Huet <charles.h...@gmail.com> a écrit :

> >* Please use Utilities/Scripts/clang-format.bash for
> >  code style.  This is easy to apply automatically later
> >  so don't worry about it too much.
> I'll get on this soon, I like to have proper style.
>
>
> > * For final integration I'd like the commits to be squashed
>
> >  and arranged in an organized way.  This is hard to do until
> >  you have everything working though.
>
> Sure. For now there are the first few commits which I can't change much to
> keep proper attribuation, and a few commits on top which will be squashed
> together once the dev is complete (or maybe earlier if I feel like it).
>
> >* Please update the license notices to use the new style that
> >  we switched to in `master`.  Both C++ and .cmake notices
> >  changed.
> Will do.
>
> Le jeu. 6 oct. 2016 à 17:32, Brad King <brad.k...@kitware.com> a écrit :
>
> On 10/06/2016 08:32 AM, Charles Huet wrote:
> > Do you think it is time to start the review, or should it wait
> > until I have 100% of the tests passing ?
>
> Thanks for the update.  Generally my reviews for new features
> mostly look at documentation, tests, style, etc. rather than
> at implementation details (which can easily be fixed/changed
> later).  Therefore final review and integration should not start
> until everything is working.
>
> However, I'm happy to take quick glances at progress once in
> a while.  Here are some comments from the current version:
>
> * Please use Utilities/Scripts/clang-format.bash for
>   code style.  This is easy to apply automatically later
>   so don't worry about it too much.
>
> * For final integration I'd like the commits to be squashed
>   and arranged in an organized way.  This is hard to do until
>   you have everything working though.
>
> * Please update the license notices to use the new style that
>   we switched to in `master`.  Both C++ and .cmake notices
>   changed.
>
> Thanks,
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] FastBuild Generator

[Charles Huet]

> It works with Ninja for me.  Are you building CMake out-of-source?
Yes.
The complete list of commands I used is (from a clean build dir):
cmake ..\CMake -G Ninja -DCMAKE_BUILD_TYPE=Debug
ninja
ctest -V -R CheckSourceTree

In a MSVC2012 x64 command prompt.
I used the installed CMake to both configure and test.

the last message is :
 CMake Error at CheckSourceTreeTest.cmake:355 (message):
   test fails: local source tree non-additions: use git add before
committing,
   or remove the files from the source tree


>I don't understand what you mean.

Like most CMake tests, the SimpleInstall test contains a CMakeLists, so the
ctest process runs cmake and builds the test.
This test has a custom_command which does 'cmake --build', so when you run
the test, the build tool will call 'camke --build', which starts the build
tool again, in the same folder hierarchy.

This is not supported by fastbuild. Since it is highly paralellized, it
uses a lock file to check if an instance is already running, in order to
avoid all the common problems you would encounter by having two instances
doing read/writes to its local database (for instance for its caching
mechanism).

Thus I am wondering how I can get this test to pass.



Le lun. 10 oct. 2016 à 19:13, Brad King <brad.k...@kitware.com> a écrit :

> On 10/10/2016 09:39 AM, Charles Huet wrote:
> > * the test CMake.CheckSourceTree does not work with Fastbuild,
> >   but I could not get it to work with Ninja either, maybe I
> >   have an environment problem ?
>
> It works with Ninja for me.  Are you building CMake out-of-source?
>
> > * the test SimpleInstall requires the build tool to be able to
> >   run multiple instances in the same hierarchy, which Fastbuild
> >   does not support.
>
> I don't understand what you mean.
>
> > * It seems SimpleInstall-Stage2 depends upon SimpleInstall,
> >   is this right ?
>
> Yes.
>
> -Brad
>
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [Discussion] Add python support for CMakeLists

[Charles Huet]

Hi,

I also worked on a similar prototype, and I think this should be discussed
further and set as a long-term goal for CMake.

The conclusions from the last time this came up were (from the top of my
head):

1)
Lua is the language that should be used, since it is easy to embed on all
the platforms CMake supports. All arguments about language X being better
than language Y do not matter, since all other languages do not answer to
the constraints that are CMake's, which are, as I understand them:
- being embeddable so as to not depend on the user to install something else
- support for lots of various platforms (e.g. ARM)

2)
This needs refactoring in CMake core that are ongoing (and some of this is
also needed for the cmake server if I understand correctly) but far from
finished.


The biggest problem I see is that there is no official stance from Kitware
on this subject.


Le mer. 11 janv. 2017 à 23:52, Shmuel H,  a écrit :

> Hello,
>
> First of all, I have been using CMake for a few years now, it is awesome
> tool, thank you.
>
> The only problem I currently have with CMake is its language, which has
> not really intended to be one. After reading a few endless discussions
> about this topic, I decided to give it a try and do something practical.
>
> My current design is using Python as an extension for the regular
> CMakeLists.txt files: if there is a CMakeLists.py file, it would be loaded.
>
> It should not be too hard to maintain, the current API design uses only
> 1-3 function that should be implemented in cmake.
>
> For more information, see the [closed] merge request #389
> .
>
> I would be happy to hear your opinion about this idea.
>
> Best regards,
> Shmuel H.
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] [RFC] Fastbuild generator

[Charles Huet]

Hi,

I ported the work of @inbilla to CMake trunk a few months ago, and fixed
quite a few tests on my fork: https://github.com/packadal/CMake

I think it would be best if we tried to synchronize our efforts, since lots
of people are interested in getting a FastBuild generator for CMake !
There is a gitter channel that is used to discuss the progress here:
https://gitter.im/inbilla/CMake

I hope we can get this to work well enough to be iontegrated in CMake 3.8,
but I did not have time to work on this lately.
I hope to get back to it this week, but I am always available for
discussion :)

Best


Le dim. 8 janv. 2017 à 05:59, comic fans  a écrit :

Hello:

Fastbuild is a opensource multi platform build tool which has built-in
cache/distribute (likes ccache/distcc) and pch/unity build
support(http://fastbuild.org/docs/home.html).


@inbilla created fastbuild generator based on 3.2.1 here
https://github.com/inbilla/CMake , this completed almost everything
but didn't continue developing for a while. recently I've rebase his
work on master branch, with some
tweaks(https://gitlab.kitware.com/comicfans/cmake/commits/upstream2),now
it can builds cmake , and almost llvm/clang,  so I'd like to share
this here, hopes more user to test this out.

I haven't touch cmake code before so it may not as good as you think.
If anyone interested to try this code, any comments and suggestion
would be helpful.


How Fastbuild compares to Ninja :
If build locally, fastbuild almost as fast as ninja , but fastbuild
provides simpler distrubte support,you can just run fastbuild worker
on many woker machine (without need to install toolchain) to speed up.
As I tested,

host (4core4thread 8GB ) local build clang:  60 min
host + worker (6core12thread 32GB) distrubite clang : 30 min

Fastbuild Generator current status:

 local build works on linux/windows (I don't have mac to test)

 some cmake variable not set correctly (RUNTIME_OUTPUT_DIRECTORY,
LIBRARY_OUTPUT_DIRECTORY,  how to put CFGIntDir in ?  help needed !),

 distrubte build not work (you can list toolchain exe/dll in header,
or just copy header from fastbuild src's to make distribute build
work) I wonder if cmake has builtin support to list all these files,
help needed!

config/build on windows must run from vs cmd console, because cmake
didn't detect windows SDK for include/link path (Ninja generator also
suffer from this). help needed.

edit CMake files didn't auto trigger fastbuild auto rebuild, must rerun
cmake

rerun cmake makes fastbuild rebuild everything. this is fastbuild's
shortcoming, author is working on this.

custom commands will generate script file (bash/bat) ,should we write
commands as argument to sh/cmd , or improve this in fastbuild ?
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Adding a new henerator for CMake

[Charles Huet]

Hi,


is this buildsystem by any chance FastBuild ?
Because if so there is an effort (I'd like to say ongoing, but it's been on
hold since the release of 9.7) to implement one here:
https://github.com/packadal/CMake/tree/fastbuild-master

Otherwise I guess it can serve as another example, as far as I know there
is no documentation on how to implement a generator from scratch.

Good luck !

Le ven. 17 mars 2017 à 02:39, Renato Utsch  a écrit :

> Hello,
>
> Is there documentation somewhere on how to add a new generator to CMake?
> There's a build system I really want CMake to support, and I'd like to
> contribute to make that happen.
>
> How difficult / how much code is involved in writing a new generator? The
> build system has nothing to do with makefiles.
>
> Thank.
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Fwd: Feature suggestion: auto-create missing files

[Charles Huet]

I personally really like the approach taken by FastBuild where you can
specify a folder in which the source resides, and it will take care of
compiling all the files within.

A checksum to verify if there are new files, and if not no need to
reconfigure.

This could pretty much be used with globing as well, making it a more
useful feature.

Also, I really appreciate the fact that you can't forget an old source file
by mistake (removed in the CMakeLists.txt but not on disk) this way.

Just my 2 cents.

On Tue, Apr 11, 2017, 19:15 Alex Turbov  wrote:

>
>
> Hi,
>
> here is my 5 cents...
>
> On Tue, Apr 11, 2017 at 10:52 PM, Brad King  wrote:
>
> On 04/11/2017 11:41 AM, Petr Kmoch wrote:
> > Currently, adding a new source file to a CMake-controlled project
> > means doing two things: creating the file on disk, and adding it
> > to the relevant CMakeList add_library() or add_executable() call.
>
> I view this as a matching pair with an implicit sanity check.
>
> +1
>
>
> > switch from current behaviour of "error out if source file is not found"
> > to "create empty source file if it's not found."
>
> So a typo in the `CMakeLists.txt` file leads to silent creation of a
> source file instead of an error message?
>
> +1
>
> doesn't looks like a good idea... also if someone (re)moved/renamed a file
> intentionally and forget to update CMakeLists.txt (or just rerun `make`
> which executes `cmake`)
>
>
> That said, I can see how the proposed feature might be useful when
> iteratively developing in an IDE.  Add the file to `CMakeLists.txt`,
> reconfigure, and open the new (now existing) file to edit in the IDE.
>
>
> my personal practice completely the opposite:
> in my CMakeLists I have a custom target to generate a source file from the
> project specific template, so I just use CLI to generate a new file like:
>
> $ make new-class name=BlahBlah ns=Vendor::Project subdir=some/dir
>
> and then go to corresponding CMakeLists.txt and add the generated
> source(s) to a target
> and thanks to `cmake` missed files are reported at configuration time
>
>
> > Is this something that would be acceptable into CMake? Any comments?
>
> I'd like to hear more opinions from others before considering it
> upstream.  It feels like a pretty personal workflow right now, and
> can be implemented in CMake code already (perhaps with the `SOURCES`
> target property to avoid separate lists).
>
> If this were to be done I'd first like to see a policy introduced to
> get rid of the magic extension guessing we do now.  Without knowing
> the full file name with confidence we wouldn't be able to create it.
>
> -Brad
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] cuda_compile_ptx re-runs CMake at build time

[Charles Huet]

Hi,

I'm having an issue with CUDA (sadly the old-style FindCUDA, not the new
native support of CUDA).

The following CMake file reproduces the issue easily on Windows with any
Visual Studio Generator (14 2015 Win64 being the one I use).

Adding the file generated by cuda_compile_ptx to any target (even a custom
one) causes CMake to be re-run at compile time.
Which is very annoying, but causes build errors on multi-threaded builds
(permission denied on the vcxproj generated by this CMakeLists.). Of
course, such issues cannot be reproduced with this simple CMakeLists file.

Am I misusing the cuda_compile_ptx method ?
Is there a way to prevent such CMake runs during compile time to occur ?

Thanks.

project(testCuda)
cmake_minimum_required(VERSION 3.10.2)
find_package(CUDA 9.2 REQUIRED)

CUDA_COMPILE_PTX(OBJECT_FILES Arithmetic.cu)

add_library(stuff SHARED SoArithmetic.cxx ${OBJECT_FILES})
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

[cmake-developers] EXCLUDE_FROM_ALL behavior change

[Charles Huet]

Hi,

in CMake 3.14 the EXCLUDE_FROM_ALL option of add_subdirectory had a
behavior change.
Here is the MR: https://gitlab.kitware.com/cmake/cmake/merge_requests/2816,
implementing the requirement from this ticket:
https://gitlab.kitware.com/cmake/cmake/issues/17149.

We are negatively impacted by this change, as we are in the same exact
scenario as this user:
https://cmake.org/pipermail/cmake/2019-September/069978.html

Except we used this behavior to have separate VS solutions for our unit
tests and demos.

Our hierarchy looks something like this:
|- MainLibrary
|- Extension1
|- Extension2
|- UnitTests
  | - [lots of tests]
|- Demos
  | - [lots of demos]

This had the very nice result of having a root solution containing all of
our libraries and extensions, another solution in the UnitTests folder, and
another in the Demos folder (and likewise on linux, we could build with
'ninja UnitTests/all').

The new behavior leads to the inability to build all our tests/demos in one
fell swoop, and removing the EXCLUDE_FROM_ALL directive would result in a
VS solution way too large for daily use.

This currently is blocking us to upgrade to CMake > 3.13, which prevents us
from using VS 2019.

I know this change occurred about two years ago, so changing the behavior
the other way is not a good way forward, and do not know enough to find out
by myself how the previous behavior can be re-introduced.

I would like advice on how to re-introduce the previous behavior without
perturbing the new one, e.g. a way to set EXCLUDE_FROM_ALL to FALSE on all
the targets of a folder that is itself excluded from the ALL target.

Thanks for your help.

-- 
 - Always keep in mind that you may be wrong -
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers

Re: [cmake-developers] EXCLUDE_FROM_ALL behavior change

[Charles Huet]

Great, thanks !

On Tue, Sep 24, 2019 at 3:38 PM Brad King  wrote:

> On 9/24/19 8:12 AM, Charles Huet wrote:
> > in CMake 3.14 the EXCLUDE_FROM_ALL option of add_subdirectory had a
> behavior change.
> > We are negatively impacted by this change, as we are in the same exact
> scenario as
> > this user: https://cmake.org/pipermail/cmake/2019-September/069978.html
>
> Thanks.  We thought the change was backward compatible but perhaps not.
> I've opened an issue:
>
>   https://gitlab.kitware.com/cmake/cmake/issues/19753
>
> -Brad
>


-- 
 - Always keep in mind that you may be wrong -
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake-developers