Re: [cmake-developers] Using CMake as a library from Python

2016-01-04 Thread Tamás Kenéz
Charles,

I'd like to comment on a minor issue:

CMakeLists.txt files don't specify user-specific data like source and build
dirs (for a reason) yet your POC listfile contains this line:

cmake.init("Ninja", "/media/dev/src/cmaketest",
"/media/dev/build/cmaketest")

which sets the source and build directories. Is this some temporary
solution for the POC only?

Tamas

On Mon, Jan 4, 2016 at 8:41 AM, Charles Huet  wrote:

> Hi,
>
> Because of the above, I worry that you are basing your work on an old
>> version of CMake. As of April 2015 cmState is used as the interface to the
>> cache.
>
>
> Yeah, that is sadly right. I wanted to rebase on master before publishing,
> thinking it should not be too hard, and I have lots to re-do, mostly
> understand how things are done now.
>
> I did something similar some years ago with QML instead of python, so it
>> sounds interesting to me.
>
>
> I'm trying to be as declarative as possible, because really like how
> readable simple QML programs are, and I think it would be perfect for a
> buildsystem.
>
> My guess is that you are using python to instantiate a cmAddLibraryCommand
>> and then executing it.
>
>
> Actually, I'm directly using the cmMakefile, because I did not want to
> wrap all the commands, and it seemed backwards to me to wrap them.
>
> Even much of cmMakefile shouldn't be used by a new language. Instead
>> certain
>> parts of cmMakefile should be extracted as other classes
>> (cmVariableExpander
>> which should have the ExpandVariablesInString and ConfigureString stuff,
>> cmMessenger for the IssueMessage stuff, some other class for the
>> CompileFeature stuff etc). Then cmMakefile would be just about executing
>> and
>> scoping the CMake language. A new language would not need that, but would
>> use the refactored extracted classes.
>
>
> Ah, this is very interesting, thanks.
>
>
> Having said all that, Brad favors Lua I believe, and he favors a different
>> approach (which no one is working on as far as I know) to adding a new
>> language. So wait to hear from him to know whether it is something that
>> would be upstreamable.
>
>
> Have any details on the approach in question ? SWIG would allow for Lua
> bindings as easily, but I don't think having multiple languages would be a
> good idea.
> I went with Python because I'm familiar with it and have already written
> bindings for it with SWIG. Also, buildbot is written in python and it could
> provide a really interesting integration I think.
>
> I would guide/support you in refactoring cmake as needed. The refactoring
>> part would definitely be upstreamable. I would very much like to see a
>> proof
>> of concept alternative language even if that wasn't upstreamed. It would
>> prove that another language is possible, and that's one of the steps to
>> replacing the current cmake language I think.
>
>
> I will need to work on it to make it work again with master, but I'll try
> and do this soon.
>
> Here is what my test POC looked like for generating a simple shared
> library:
>
> #!/usr/bin/env python
>> # -*- coding: utf-8 -*-
>> import cmake
>> cmake.init("Ninja", "/media/dev/src/cmaketest",
>> "/media/dev/build/cmaketest")
>> myProject = cmake.Project("MyTestProject")
>> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
>> cmake.generate()
>
>
> Thanks a lot for your comments, I'm happy to see this was not just a dumb
> idea, and that others have thought about it.
>
> I'll update with the github as soon as I get it working.
>
> Best
>
>
> Le lun. 4 janv. 2016 à 01:16, Stephen Kelly  a écrit :
>
>> Charles Huet wrote:
>>
>> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
>> > cannot be used from python (it check for the existence of a
>> CMakeLists.txt
>> > file, which does not exist in this scenario) and the cache variables it
>> > sets seem to be necessary.
>>
>> Because of the above, I worry that you are basing your work on an old
>> version of CMake. As of April 2015 cmState is used as the interface to the
>> cache.
>>
>>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13
>>
>> Also note that the C++ interfaces in CMake are not stable. In the last
>> year
>> they have been changed utterly and that will continue to happen without
>> notice. I'm sure you know this, but I thought I'd say it anyway.
>>
>> > I will try to make a layer of abstraction on top of the python bindings
>> > before publishing this work on github, in order to show the syntactic
>> > sugar python can provide, but I will publish this before if anybody is
>> > interested in working on this.
>>
>> I would be interested in seeing it.
>>
>> > Now, does anyone beside me think this is a good idea ?
>>
>> I did something similar some years ago with QML instead of python, so it
>> sounds interesting to me.
>>
>> In fact, one of the reasons for introducing cmState and doing all the
>> refactoring I did in cmake was to make it possible to 

Re: [cmake-developers] Using CMake as a library from Python

2016-01-04 Thread Charles Huet
I took the time to make it work again, so I pushed it here:
https://github.com/packadal/CMake/tree/cmake-python

The whole branch is ugly as can be, and because I started with an old CMake
and recently rebased, the python abstraction might be bloated already, but
it works in the nominal case, and it would be pretty easy to add most of
the CMake basic functionalities.

I am not used to writing python, so it might not be idiomatic either, but
hey, it works as a simple proof of concept, and it's all I wanted to do for
now :)

Le lun. 4 janv. 2016 à 08:41, Charles Huet  a
écrit :

> Hi,
>
> Because of the above, I worry that you are basing your work on an old
>> version of CMake. As of April 2015 cmState is used as the interface to the
>> cache.
>
>
> Yeah, that is sadly right. I wanted to rebase on master before publishing,
> thinking it should not be too hard, and I have lots to re-do, mostly
> understand how things are done now.
>
> I did something similar some years ago with QML instead of python, so it
>> sounds interesting to me.
>
>
> I'm trying to be as declarative as possible, because really like how
> readable simple QML programs are, and I think it would be perfect for a
> buildsystem.
>
> My guess is that you are using python to instantiate a cmAddLibraryCommand
>> and then executing it.
>
>
> Actually, I'm directly using the cmMakefile, because I did not want to
> wrap all the commands, and it seemed backwards to me to wrap them.
>
> Even much of cmMakefile shouldn't be used by a new language. Instead
>> certain
>> parts of cmMakefile should be extracted as other classes
>> (cmVariableExpander
>> which should have the ExpandVariablesInString and ConfigureString stuff,
>> cmMessenger for the IssueMessage stuff, some other class for the
>> CompileFeature stuff etc). Then cmMakefile would be just about executing
>> and
>> scoping the CMake language. A new language would not need that, but would
>> use the refactored extracted classes.
>
>
> Ah, this is very interesting, thanks.
>
>
> Having said all that, Brad favors Lua I believe, and he favors a different
>> approach (which no one is working on as far as I know) to adding a new
>> language. So wait to hear from him to know whether it is something that
>> would be upstreamable.
>
>
> Have any details on the approach in question ? SWIG would allow for Lua
> bindings as easily, but I don't think having multiple languages would be a
> good idea.
> I went with Python because I'm familiar with it and have already written
> bindings for it with SWIG. Also, buildbot is written in python and it could
> provide a really interesting integration I think.
>
> I would guide/support you in refactoring cmake as needed. The refactoring
>> part would definitely be upstreamable. I would very much like to see a
>> proof
>> of concept alternative language even if that wasn't upstreamed. It would
>> prove that another language is possible, and that's one of the steps to
>> replacing the current cmake language I think.
>
>
> I will need to work on it to make it work again with master, but I'll try
> and do this soon.
>
> Here is what my test POC looked like for generating a simple shared
> library:
>
> #!/usr/bin/env python
>> # -*- coding: utf-8 -*-
>> import cmake
>> cmake.init("Ninja", "/media/dev/src/cmaketest",
>> "/media/dev/build/cmaketest")
>> myProject = cmake.Project("MyTestProject")
>> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
>> cmake.generate()
>
>
> Thanks a lot for your comments, I'm happy to see this was not just a dumb
> idea, and that others have thought about it.
>
> I'll update with the github as soon as I get it working.
>
> Best
>
>
> Le lun. 4 janv. 2016 à 01:16, Stephen Kelly  a écrit :
>
>> Charles Huet wrote:
>>
>> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
>> > cannot be used from python (it check for the existence of a
>> CMakeLists.txt
>> > file, which does not exist in this scenario) and the cache variables it
>> > sets seem to be necessary.
>>
>> Because of the above, I worry that you are basing your work on an old
>> version of CMake. As of April 2015 cmState is used as the interface to the
>> cache.
>>
>>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13
>>
>> Also note that the C++ interfaces in CMake are not stable. In the last
>> year
>> they have been changed utterly and that will continue to happen without
>> notice. I'm sure you know this, but I thought I'd say it anyway.
>>
>> > I will try to make a layer of abstraction on top of the python bindings
>> > before publishing this work on github, in order to show the syntactic
>> > sugar python can provide, but I will publish this before if anybody is
>> > interested in working on this.
>>
>> I would be interested in seeing it.
>>
>> > Now, does anyone beside me think this is a good idea ?
>>
>> I did something similar some years ago with QML instead of python, so it
>> 

Re: [cmake-developers] Using CMake as a library from Python

2016-01-04 Thread Charles Huet
Yes, this is POC-only.

I felt too lazy to make an argparse object and properly perform these tasks.
Obviously, the generator and source dir should be arguments.
The build dir should be the execution dir of the script, to mimic CMake
behavior.

This should not be difficult, but as I said I wanted first to get this
disussion started.

Le lun. 4 janv. 2016 à 11:54, Tamás Kenéz  a écrit :

> Charles,
>
> I'd like to comment on a minor issue:
>
> CMakeLists.txt files don't specify user-specific data like source and
> build dirs (for a reason) yet your POC listfile contains this line:
>
> cmake.init("Ninja", "/media/dev/src/cmaketest",
> "/media/dev/build/cmaketest")
>
> which sets the source and build directories. Is this some temporary
> solution for the POC only?
>
> Tamas
>
> On Mon, Jan 4, 2016 at 8:41 AM, Charles Huet 
> wrote:
>
>> Hi,
>>
>> Because of the above, I worry that you are basing your work on an old
>>> version of CMake. As of April 2015 cmState is used as the interface to
>>> the
>>> cache.
>>
>>
>> Yeah, that is sadly right. I wanted to rebase on master before
>> publishing, thinking it should not be too hard, and I have lots to re-do,
>> mostly understand how things are done now.
>>
>> I did something similar some years ago with QML instead of python, so it
>>> sounds interesting to me.
>>
>>
>> I'm trying to be as declarative as possible, because really like how
>> readable simple QML programs are, and I think it would be perfect for a
>> buildsystem.
>>
>> My guess is that you are using python to instantiate a cmAddLibraryCommand
>>> and then executing it.
>>
>>
>> Actually, I'm directly using the cmMakefile, because I did not want to
>> wrap all the commands, and it seemed backwards to me to wrap them.
>>
>> Even much of cmMakefile shouldn't be used by a new language. Instead
>>> certain
>>> parts of cmMakefile should be extracted as other classes
>>> (cmVariableExpander
>>> which should have the ExpandVariablesInString and ConfigureString stuff,
>>> cmMessenger for the IssueMessage stuff, some other class for the
>>> CompileFeature stuff etc). Then cmMakefile would be just about executing
>>> and
>>> scoping the CMake language. A new language would not need that, but would
>>> use the refactored extracted classes.
>>
>>
>> Ah, this is very interesting, thanks.
>>
>>
>> Having said all that, Brad favors Lua I believe, and he favors a different
>>> approach (which no one is working on as far as I know) to adding a new
>>> language. So wait to hear from him to know whether it is something that
>>> would be upstreamable.
>>
>>
>> Have any details on the approach in question ? SWIG would allow for Lua
>> bindings as easily, but I don't think having multiple languages would be a
>> good idea.
>> I went with Python because I'm familiar with it and have already written
>> bindings for it with SWIG. Also, buildbot is written in python and it could
>> provide a really interesting integration I think.
>>
>> I would guide/support you in refactoring cmake as needed. The refactoring
>>> part would definitely be upstreamable. I would very much like to see a
>>> proof
>>> of concept alternative language even if that wasn't upstreamed. It would
>>> prove that another language is possible, and that's one of the steps to
>>> replacing the current cmake language I think.
>>
>>
>> I will need to work on it to make it work again with master, but I'll try
>> and do this soon.
>>
>> Here is what my test POC looked like for generating a simple shared
>> library:
>>
>> #!/usr/bin/env python
>>> # -*- coding: utf-8 -*-
>>> import cmake
>>> cmake.init("Ninja", "/media/dev/src/cmaketest",
>>> "/media/dev/build/cmaketest")
>>> myProject = cmake.Project("MyTestProject")
>>> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
>>> cmake.generate()
>>
>>
>> Thanks a lot for your comments, I'm happy to see this was not just a dumb
>> idea, and that others have thought about it.
>>
>> I'll update with the github as soon as I get it working.
>>
>> Best
>>
>>
>> Le lun. 4 janv. 2016 à 01:16, Stephen Kelly  a
>> écrit :
>>
>>> Charles Huet wrote:
>>>
>>> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
>>> > cannot be used from python (it check for the existence of a
>>> CMakeLists.txt
>>> > file, which does not exist in this scenario) and the cache variables it
>>> > sets seem to be necessary.
>>>
>>> Because of the above, I worry that you are basing your work on an old
>>> version of CMake. As of April 2015 cmState is used as the interface to
>>> the
>>> cache.
>>>
>>>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13
>>>
>>> Also note that the C++ interfaces in CMake are not stable. In the last
>>> year
>>> they have been changed utterly and that will continue to happen without
>>> notice. I'm sure you know this, but I thought I'd say it anyway.
>>>
>>> > I will try to make a layer of abstraction 

Re: [cmake-developers] Using CMake as a library from Python

2016-01-03 Thread Stephen Kelly
Charles Huet wrote:

> * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
> cannot be used from python (it check for the existence of a CMakeLists.txt
> file, which does not exist in this scenario) and the cache variables it
> sets seem to be necessary.

Because of the above, I worry that you are basing your work on an old 
version of CMake. As of April 2015 cmState is used as the interface to the 
cache.

 https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13

Also note that the C++ interfaces in CMake are not stable. In the last year 
they have been changed utterly and that will continue to happen without 
notice. I'm sure you know this, but I thought I'd say it anyway.

> I will try to make a layer of abstraction on top of the python bindings
> before publishing this work on github, in order to show the syntactic
> sugar python can provide, but I will publish this before if anybody is
> interested in working on this.

I would be interested in seeing it.

> Now, does anyone beside me think this is a good idea ?

I did something similar some years ago with QML instead of python, so it 
sounds interesting to me. 

In fact, one of the reasons for introducing cmState and doing all the 
refactoring I did in cmake was to make it possible to cleanly replace the 
language, some day. 

My guess is that you are using python to instantiate a cmAddLibraryCommand 
and then executing it. I think a better approach would be to leave all the 
cm*Command classes behind as they carry a lot of backward compatibility 
baggage and policies which a new language shouldn't be burdened with. 

Even much of cmMakefile shouldn't be used by a new language. Instead certain 
parts of cmMakefile should be extracted as other classes (cmVariableExpander 
which should have the ExpandVariablesInString and ConfigureString stuff, 
cmMessenger for the IssueMessage stuff, some other class for the 
CompileFeature stuff etc). Then cmMakefile would be just about executing and 
scoping the CMake language. A new language would not need that, but would 
use the refactored extracted classes.

So, you would implement new cmPython*Command or whatever which operate on 
those classes and cmState. cmState is designed to be a language-agnostic 
store of data, like a database of buildsystem state.

As I said though, this would require further refactoring of the cmake code. 
I can provide guidance on how to do that if you are interested in pursuing 
that route. It would take some months I think, but be very valuable for many 
reasons and long-term cmake features. Another example of the kind of 
refactoring I mean is putting target state in cmState and making it 
accessible through cmState::Target, similar to how cmState::Directory 
currently works.

Having said all that, Brad favors Lua I believe, and he favors a different 
approach (which no one is working on as far as I know) to adding a new 
language. So wait to hear from him to know whether it is something that 
would be upstreamable.

I would prefer an approach similar to what I described above, which is close 
to what you are doing, so as a proof of concept I would like to see your 
work. 

I would guide/support you in refactoring cmake as needed. The refactoring 
part would definitely be upstreamable. I would very much like to see a proof 
of concept alternative language even if that wasn't upstreamed. It would 
prove that another language is possible, and that's one of the steps to 
replacing the current cmake language I think.

Thanks,

Steve.


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers


Re: [cmake-developers] Using CMake as a library from Python

2016-01-03 Thread Charles Huet
Hi,

Because of the above, I worry that you are basing your work on an old
> version of CMake. As of April 2015 cmState is used as the interface to the
> cache.


Yeah, that is sadly right. I wanted to rebase on master before publishing,
thinking it should not be too hard, and I have lots to re-do, mostly
understand how things are done now.

I did something similar some years ago with QML instead of python, so it
> sounds interesting to me.


I'm trying to be as declarative as possible, because really like how
readable simple QML programs are, and I think it would be perfect for a
buildsystem.

My guess is that you are using python to instantiate a cmAddLibraryCommand
> and then executing it.


Actually, I'm directly using the cmMakefile, because I did not want to wrap
all the commands, and it seemed backwards to me to wrap them.

Even much of cmMakefile shouldn't be used by a new language. Instead certain
> parts of cmMakefile should be extracted as other classes
> (cmVariableExpander
> which should have the ExpandVariablesInString and ConfigureString stuff,
> cmMessenger for the IssueMessage stuff, some other class for the
> CompileFeature stuff etc). Then cmMakefile would be just about executing
> and
> scoping the CMake language. A new language would not need that, but would
> use the refactored extracted classes.


Ah, this is very interesting, thanks.


Having said all that, Brad favors Lua I believe, and he favors a different
> approach (which no one is working on as far as I know) to adding a new
> language. So wait to hear from him to know whether it is something that
> would be upstreamable.


Have any details on the approach in question ? SWIG would allow for Lua
bindings as easily, but I don't think having multiple languages would be a
good idea.
I went with Python because I'm familiar with it and have already written
bindings for it with SWIG. Also, buildbot is written in python and it could
provide a really interesting integration I think.

I would guide/support you in refactoring cmake as needed. The refactoring
> part would definitely be upstreamable. I would very much like to see a
> proof
> of concept alternative language even if that wasn't upstreamed. It would
> prove that another language is possible, and that's one of the steps to
> replacing the current cmake language I think.


I will need to work on it to make it work again with master, but I'll try
and do this soon.

Here is what my test POC looked like for generating a simple shared library:

#!/usr/bin/env python
> # -*- coding: utf-8 -*-
> import cmake
> cmake.init("Ninja", "/media/dev/src/cmaketest",
> "/media/dev/build/cmaketest")
> myProject = cmake.Project("MyTestProject")
> myProject.targets = [ cmake.SharedLibrary("testLibrary", ["lib.cxx"]) ]
> cmake.generate()


Thanks a lot for your comments, I'm happy to see this was not just a dumb
idea, and that others have thought about it.

I'll update with the github as soon as I get it working.

Best


Le lun. 4 janv. 2016 à 01:16, Stephen Kelly  a écrit :

> Charles Huet wrote:
>
> > * cmCacheManager::AddCacheEntry was made public, as cmake::Configure
> > cannot be used from python (it check for the existence of a
> CMakeLists.txt
> > file, which does not exist in this scenario) and the cache variables it
> > sets seem to be necessary.
>
> Because of the above, I worry that you are basing your work on an old
> version of CMake. As of April 2015 cmState is used as the interface to the
> cache.
>
>  https://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a6b1ad13
>
> Also note that the C++ interfaces in CMake are not stable. In the last year
> they have been changed utterly and that will continue to happen without
> notice. I'm sure you know this, but I thought I'd say it anyway.
>
> > I will try to make a layer of abstraction on top of the python bindings
> > before publishing this work on github, in order to show the syntactic
> > sugar python can provide, but I will publish this before if anybody is
> > interested in working on this.
>
> I would be interested in seeing it.
>
> > Now, does anyone beside me think this is a good idea ?
>
> I did something similar some years ago with QML instead of python, so it
> sounds interesting to me.
>
> In fact, one of the reasons for introducing cmState and doing all the
> refactoring I did in cmake was to make it possible to cleanly replace the
> language, some day.
>
> My guess is that you are using python to instantiate a cmAddLibraryCommand
> and then executing it. I think a better approach would be to leave all the
> cm*Command classes behind as they carry a lot of backward compatibility
> baggage and policies which a new language shouldn't be burdened with.
>
> Even much of cmMakefile shouldn't be used by a new language. Instead
> certain
> parts of cmMakefile should be extracted as other classes
> (cmVariableExpander
> which should have the ExpandVariablesInString and ConfigureString stuff,
> cmMessenger for the