Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 05/17/2011 10:43 AM, Manuel Klimek wrote: Wow. Well, the language (and thus the stl) and the compilers have evolved - sometimes it's a shame if one has to support old versions that even the compiler writers would consider broken, but there is no choice - thanks for showing those corner cases, they're definitely educating for me. More porting: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=4268e3d7 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=7039d1fd -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Tue, May 17, 2011 at 6:03 AM, Brad King brad.k...@kitware.com wrote: On 05/16/2011 12:37 PM, Manuel Klimek wrote: On Mon, May 16, 2011 at 7:58 AM, Brad King brad.k...@kitware.com wrote: On 05/14/2011 01:04 AM, Manuel Klimek wrote: Any news on this one? We're currently building more tools, so it would be cool to know when we can expect this to be more easily available. If there's more work to do on my side, I'm happy to see it through. Thanks for the ping. I've had the topic sitting in my work tree for a while but needed to make one more fix. I just needed to tweak your last commit to only build the test helper code if the test would be run. I've merged the topic to our 'next' branch: http://cmake.org/gitweb?p=cmake.git;a=commit;h=024f2cdc FYI, I had to fix several errors to make this topic portable: http://www.cdash.org/CDash/index.php?project=CMakefiltercount=3showfilters=1filtercombine=andfield1=buildstarttime/datecompare1=83value1=2011-05-16%2021%3A00%3A00%20EDTfield2=buildstarttime/datecompare2=84value2=2011-05-17%2006%3A00%3A00%20EDTfield3=builderrors/numbercompare3=43value3=0 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c9174c0e http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=a7e7a04a http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=4e2185cb http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=169bb05f It's amazing how little of the STL can actually be used ;) Wow. Well, the language (and thus the stl) and the compilers have evolved - sometimes it's a shame if one has to support old versions that even the compiler writers would consider broken, but there is no choice - thanks for showing those corner cases, they're definitely educating for me. Cheers, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 05/14/2011 01:04 AM, Manuel Klimek wrote: Any news on this one? We're currently building more tools, so it would be cool to know when we can expect this to be more easily available. If there's more work to do on my side, I'm happy to see it through. Thanks for the ping. I've had the topic sitting in my work tree for a while but needed to make one more fix. I just needed to tweak your last commit to only build the test helper code if the test would be run. I've merged the topic to our 'next' branch: http://cmake.org/gitweb?p=cmake.git;a=commit;h=024f2cdc Thanks for your work, -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Mon, May 16, 2011 at 7:58 AM, Brad King brad.k...@kitware.com wrote: On 05/14/2011 01:04 AM, Manuel Klimek wrote: Any news on this one? We're currently building more tools, so it would be cool to know when we can expect this to be more easily available. If there's more work to do on my side, I'm happy to see it through. Thanks for the ping. I've had the topic sitting in my work tree for a while but needed to make one more fix. I just needed to tweak your last commit to only build the test helper code if the test would be run. I've merged the topic to our 'next' branch: http://cmake.org/gitweb?p=cmake.git;a=commit;h=024f2cdc Cool, thanks! /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 04/05/2011 12:13 PM, Manuel Klimek wrote: On Tue, Apr 5, 2011 at 6:19 AM, Brad King brad.k...@kitware.com wrote: On 04/05/2011 08:33 AM, Brad King wrote: On 04/04/2011 12:01 PM, Manuel Klimek wrote: Done. Thanks! While running the test on more platforms I ran into a problem. On Windows makefile tools like Borland and NMake we use custom inline response file syntax to handle long command lines: http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/Windows.cmake;hb=v2.8.4#l16 This syntax ends up in compile_commands.json and then makes no sense. This opens a can of worms. Unless we can refactor the generators to avoid using inline response file syntax then this whole feature will not work. Recently I added support for using cmake-generated response files for -I paths on some platforms: http://cmake.org/gitweb?p=cmake.git;a=commit;h=1c3233a8 It is a start, but I have no time to work further on ensuring command lines in the makefiles stay short enough for the ancient Windows make tools. For what platforms do you actually need this feature? The first version is completely fine as unix-only - clang on Windows is not exactly there yet, so I think doing the Windows stuff as a second step when we have proven the usefulness in unix land by integrating some tools will make sense. I still cannot integrate this patch series until the above problems have been addressed. Please extend the series with a patch that disables the option except on the platforms where you know it works. Thanks, -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Mon, Apr 25, 2011 at 8:27 AM, Brad King brad.k...@kitware.com wrote: On 04/05/2011 12:13 PM, Manuel Klimek wrote: On Tue, Apr 5, 2011 at 6:19 AM, Brad King brad.k...@kitware.com wrote: On 04/05/2011 08:33 AM, Brad King wrote: On 04/04/2011 12:01 PM, Manuel Klimek wrote: Done. Thanks! While running the test on more platforms I ran into a problem. On Windows makefile tools like Borland and NMake we use custom inline response file syntax to handle long command lines: http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/Windows.cmake;hb=v2.8.4#l16 This syntax ends up in compile_commands.json and then makes no sense. This opens a can of worms. Unless we can refactor the generators to avoid using inline response file syntax then this whole feature will not work. Recently I added support for using cmake-generated response files for -I paths on some platforms: http://cmake.org/gitweb?p=cmake.git;a=commit;h=1c3233a8 It is a start, but I have no time to work further on ensuring command lines in the makefiles stay short enough for the ancient Windows make tools. For what platforms do you actually need this feature? The first version is completely fine as unix-only - clang on Windows is not exactly there yet, so I think doing the Windows stuff as a second step when we have proven the usefulness in unix land by integrating some tools will make sense. I still cannot integrate this patch series until the above problems have been addressed. Please extend the series with a patch that disables the option except on the platforms where you know it works. Is just putting IF(UNIX) ... around all the stuff enough for that? Thanks, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 04/25/2011 11:54 AM, Manuel Klimek wrote: Is just putting IF(UNIX) ... around all the stuff enough for that? It's not just the test. The C++ implementation cannot handle all generators. It should be taught to run only for generators where it is known to work. Perhaps it can check a value set by each platform module. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Mon, Apr 25, 2011 at 8:59 AM, Brad King brad.k...@kitware.com wrote: On 04/25/2011 11:54 AM, Manuel Klimek wrote: Is just putting IF(UNIX) ... around all the stuff enough for that? It's not just the test. The C++ implementation cannot handle all generators. It should be taught to run only for generators where it is known to work. Perhaps it can check a value set by each platform module. Ah, ok, I was thinking putting it around the exporting of the flag, so that you can only switch it on on supported platforms - as far as I can tell, the code still works on other platforms, the database is just not enough, right? Cheers, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 04/25/2011 12:06 PM, Manuel Klimek wrote: On Mon, Apr 25, 2011 at 8:59 AM, Brad King brad.k...@kitware.com wrote: On 04/25/2011 11:54 AM, Manuel Klimek wrote: Is just putting IF(UNIX) ... around all the stuff enough for that? It's not just the test. The C++ implementation cannot handle all generators. It should be taught to run only for generators where it is known to work. Perhaps it can check a value set by each platform module. Ah, ok, I was thinking putting it around the exporting of the flag, so that you can only switch it on on supported platforms That approach may work. It is up to the platform modules to present the option when it can be implemented. If that is just a few conditions around a single place then great. - as far as I can tell, the code still works on other platforms, the database is just not enough, right? The main problem is that some of the generators write syntax that is specific to the make tool (e.g. NMake, Borland) directly into the command line. That syntax confuses any tool that wants to read it from the database. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 04/06/2011 10:15 PM, Manuel Klimek wrote: For your convenience, rebased onto current head and git format-patch'ed. Thanks. I used that series to reconstruct the topic on master locally. I will review and test further when I get a chance. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
For your convenience, rebased onto current head and git format-patch'ed.
Cheers,
/Manuel
On Tue, Apr 5, 2011 at 9:14 AM, Manuel Klimek kli...@google.com wrote:
On Tue, Apr 5, 2011 at 5:33 AM, Brad King brad.k...@kitware.com wrote:
On 04/04/2011 12:01 PM, Manuel Klimek wrote:
Done.
Thanks!
I also have the previous patches ready (re-based with master). Let me
know how we'll go on about submitting all that.
I still have them on a local topic branch. See below.
Have you changed anything since then?
Well, the git rebase led to conflicts which I resolved - otherwise I
didn't change anything on the other patches. So I can re-export the
changes to save you some work with merging, if you want :)
Cheers,
/Manuel
If you prefer to send the new versions, please use git format-patch to
construct the patch series. Then send the files by email.
Thanks,
-Brad
commit c8ed16a373adf98cf21a619349de8c0bde6a0eed
Author: Manuel Klimek kli...@google.com
Date: Mon Jan 17 11:44:44 2011 -0500
Make compile line export optional
Add option CMAKE_EXPORT_COMPILE_COMMANDS to control the behavior.
Modules/CMakeGenericSystem.cmake | 3 +++
Source/cmGlobalUnixMakefileGenerator3.cxx | 2 +-
Source/cmMakefileTargetGenerator.cxx | 3 ++-
3 files changed, 6 insertions(+), 2 deletions(-)
commit 357fdbb1d7c472991323d36c6a431a36f16918bd
Author: Manuel Klimek kli...@google.com
Date: Thu Jan 13 16:15:54 2011 -0500
Export all compile command lines to json database
Source/cmGlobalUnixMakefileGenerator3.cxx | 41
+
Source/cmGlobalUnixMakefileGenerator3.h | 6
Source/cmMakefileTargetGenerator.cxx | 21 +-
3 files changed, 66 insertions(+), 2 deletions(-)
commit a75e21cadeb48c257d870db925131994f7da4995
Author: Manuel Klimek kli...@google.com
Date: Thu Jan 13 16:06:20 2011 -0500
Factor Makefile flag and define lookup into methods
Cache results over repeated calls.
Source/cmMakefileTargetGenerator.cxx | 107
+++--
Source/cmMakefileTargetGenerator.h | 5 ++
2 files changed, 67 insertions(+), 45 deletions(-)
--
Manuel Klimek (http://go/klimek)
From 8b304e3c4a111a242033d261adb9f02f116cfa96 Mon Sep 17 00:00:00 2001
From: Manuel Klimek kli...@google.com
Date: Wed, 12 Jan 2011 20:06:25 -0800
Subject: [PATCH 1/5] refactor flags and defines
---
Source/cmMakefileTargetGenerator.cxx | 120 ++---
Source/cmMakefileTargetGenerator.h |3 +
2 files changed, 68 insertions(+), 55 deletions(-)
diff --git a/Source/cmMakefileTargetGenerator.cxx b/Source/cmMakefileTargetGenerator.cxx
index 69320da..01545fc 100644
--- a/Source/cmMakefileTargetGenerator.cxx
+++ b/Source/cmMakefileTargetGenerator.cxx
@@ -249,6 +249,66 @@ void cmMakefileTargetGenerator::WriteCommonCodeRules()
}
//
+std::string cmMakefileTargetGenerator::GetFlags(const std::string l) {
+ std::string flags;
+ const char *lang = l.c_str();
+
+ bool shared = ((this-Target-GetType() == cmTarget::SHARED_LIBRARY) ||
+ (this-Target-GetType() == cmTarget::MODULE_LIBRARY));
+
+ // Add language feature flags.
+ this-AddFeatureFlags(flags, lang);
+
+ this-LocalGenerator-AddArchitectureFlags(flags, this-Target,
+ lang, this-ConfigName);
+
+ // Fortran-specific flags computed for this target.
+ if(l == Fortran)
+{
+this-AddFortranFlags(flags);
+}
+
+ // Add shared-library flags if needed.
+ this-LocalGenerator-AddSharedFlags(flags, lang, shared);
+
+ // Add include directory flags.
+ this-AddIncludeFlags(flags, lang);
+
+ // Append old-style preprocessor definition flags.
+ this-LocalGenerator-
+AppendFlags(flags, this-Makefile-GetDefineFlags());
+
+ // Add include directory flags.
+ this-LocalGenerator-
+AppendFlags(flags,this-GetFrameworkFlags().c_str());
+
+ return flags;
+}
+
+std::string cmMakefileTargetGenerator::GetDefines(const std::string l) {
+ std::string defines;
+ const char *lang = l.c_str();
+ // Add the export symbol definition for shared library objects.
+ if(const char* exportMacro = this-Target-GetExportMacro())
+{
+this-LocalGenerator-AppendDefines(defines, exportMacro, lang);
+}
+
+ // Add preprocessor definitions for this target and configuration.
+ this-LocalGenerator-AppendDefines
+(defines, this-Makefile-GetProperty(COMPILE_DEFINITIONS), lang);
+ this-LocalGenerator-AppendDefines
+(defines, this-Target-GetProperty(COMPILE_DEFINITIONS), lang);
+ std::string defPropName = COMPILE_DEFINITIONS_;
+ defPropName +=
+cmSystemTools::UpperCase(this-LocalGenerator-ConfigurationName);
+ this-LocalGenerator-AppendDefines
+(defines, this-Makefile-GetProperty(defPropName.c_str()), lang);
+ this-LocalGenerator-AppendDefines
+(defines,
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 04/04/2011 12:01 PM, Manuel Klimek wrote: Done. Thanks! I also have the previous patches ready (re-based with master). Let me know how we'll go on about submitting all that. I still have them on a local topic branch. See below. Have you changed anything since then? If you prefer to send the new versions, please use git format-patch to construct the patch series. Then send the files by email. Thanks, -Brad commit c8ed16a373adf98cf21a619349de8c0bde6a0eed Author: Manuel Klimek kli...@google.com Date: Mon Jan 17 11:44:44 2011 -0500 Make compile line export optional Add option CMAKE_EXPORT_COMPILE_COMMANDS to control the behavior. Modules/CMakeGenericSystem.cmake |3 +++ Source/cmGlobalUnixMakefileGenerator3.cxx |2 +- Source/cmMakefileTargetGenerator.cxx |3 ++- 3 files changed, 6 insertions(+), 2 deletions(-) commit 357fdbb1d7c472991323d36c6a431a36f16918bd Author: Manuel Klimek kli...@google.com Date: Thu Jan 13 16:15:54 2011 -0500 Export all compile command lines to json database Source/cmGlobalUnixMakefileGenerator3.cxx | 41 + Source/cmGlobalUnixMakefileGenerator3.h |6 Source/cmMakefileTargetGenerator.cxx | 21 +- 3 files changed, 66 insertions(+), 2 deletions(-) commit a75e21cadeb48c257d870db925131994f7da4995 Author: Manuel Klimek kli...@google.com Date: Thu Jan 13 16:06:20 2011 -0500 Factor Makefile flag and define lookup into methods Cache results over repeated calls. Source/cmMakefileTargetGenerator.cxx | 107 +++-- Source/cmMakefileTargetGenerator.h |5 ++ 2 files changed, 67 insertions(+), 45 deletions(-) ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Tue, Apr 5, 2011 at 6:19 AM, Brad King brad.k...@kitware.com wrote: On 04/05/2011 08:33 AM, Brad King wrote: On 04/04/2011 12:01 PM, Manuel Klimek wrote: Done. Thanks! While running the test on more platforms I ran into a problem. On Windows makefile tools like Borland and NMake we use custom inline response file syntax to handle long command lines: http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/Platform/Windows.cmake;hb=v2.8.4#l16 This syntax ends up in compile_commands.json and then makes no sense. This opens a can of worms. Unless we can refactor the generators to avoid using inline response file syntax then this whole feature will not work. Recently I added support for using cmake-generated response files for -I paths on some platforms: http://cmake.org/gitweb?p=cmake.git;a=commit;h=1c3233a8 It is a start, but I have no time to work further on ensuring command lines in the makefiles stay short enough for the ancient Windows make tools. For what platforms do you actually need this feature? The first version is completely fine as unix-only - clang on Windows is not exactly there yet, so I think doing the Windows stuff as a second step when we have proven the usefulness in unix land by integrating some tools will make sense. Cheers, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Tue, Apr 5, 2011 at 5:33 AM, Brad King brad.k...@kitware.com wrote: On 04/04/2011 12:01 PM, Manuel Klimek wrote: Done. Thanks! I also have the previous patches ready (re-based with master). Let me know how we'll go on about submitting all that. I still have them on a local topic branch. See below. Have you changed anything since then? Well, the git rebase led to conflicts which I resolved - otherwise I didn't change anything on the other patches. So I can re-export the changes to save you some work with merging, if you want :) Cheers, /Manuel If you prefer to send the new versions, please use git format-patch to construct the patch series. Then send the files by email. Thanks, -Brad commit c8ed16a373adf98cf21a619349de8c0bde6a0eed Author: Manuel Klimek kli...@google.com Date: Mon Jan 17 11:44:44 2011 -0500 Make compile line export optional Add option CMAKE_EXPORT_COMPILE_COMMANDS to control the behavior. Modules/CMakeGenericSystem.cmake | 3 +++ Source/cmGlobalUnixMakefileGenerator3.cxx | 2 +- Source/cmMakefileTargetGenerator.cxx | 3 ++- 3 files changed, 6 insertions(+), 2 deletions(-) commit 357fdbb1d7c472991323d36c6a431a36f16918bd Author: Manuel Klimek kli...@google.com Date: Thu Jan 13 16:15:54 2011 -0500 Export all compile command lines to json database Source/cmGlobalUnixMakefileGenerator3.cxx | 41 + Source/cmGlobalUnixMakefileGenerator3.h | 6 Source/cmMakefileTargetGenerator.cxx | 21 +- 3 files changed, 66 insertions(+), 2 deletions(-) commit a75e21cadeb48c257d870db925131994f7da4995 Author: Manuel Klimek kli...@google.com Date: Thu Jan 13 16:06:20 2011 -0500 Factor Makefile flag and define lookup into methods Cache results over repeated calls. Source/cmMakefileTargetGenerator.cxx | 107 +++-- Source/cmMakefileTargetGenerator.h | 5 ++ 2 files changed, 67 insertions(+), 45 deletions(-) -- Manuel Klimek (http://go/klimek) ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 04/01/2011 02:42 PM, Manuel Klimek wrote:
sorry for the delay, but I was busy with other stuff - please find
attached a proposal patch for the test - I don't know enough about the
cmake testing infrastructure to say whether that was what you
imagined, please let me know if I should change anything.
That looks pretty good, thanks. Here are a few comments from review:
- The cmSystemTools::ParseArguments method is not very solid. It fails for
double-quotes in the middle of an argument, like '-Ia b'. Use the newer
ParseUnixCommandLine instead. Unfortunately for historical reasons we
need to convert from vectorstd::string to vectorcmStdString. Try
the patch below.
- The Source directory should not be touched for this. Put the source code
in Tests/CMakeLib and move the code to build the executable to the
CMakeLists.txt file in that directory.
- In the CompileCommandOutput test source files, please actually create a
symbol in each library and call them from the main. Otherwise some
toolchains will complain about empty object files.
- The ADD_TEST_MACRO call will need to be conditioned to run only when the
exported compile lines will work. I will take care of that for you when
the rest of the patch is ready.
Thanks,
-Brad
diff --git a/Source/run_compile_commands.cxx b/Source/run_compile_commands.cxx
index 0b248f7..82cedfb 100644
--- a/Source/run_compile_commands.cxx
+++ b/Source/run_compile_commands.cxx
@@ -117,8 +117,9 @@ int main ()
it = parser.GetTranslationUnits().begin(),
end = parser.GetTranslationUnits().end(); it != end; ++it)
{
-std::vectorcmStdString command =
-cmSystemTools::ParseArguments(it-at(command).c_str());
+std::vectorstd::string command1;
+cmSystemTools::ParseUnixCommandLine(it-at(command).c_str(), command1);
+std::vectorcmStdString command(command1.begin(), command1.end());
if (!cmSystemTools::RunSingleCommand(
command, 0, 0, it-at(directory).c_str()))
{
___
cmake-developers mailing list
cmake-developers@cmake.org
http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
Hi Brad, sorry for the delay, but I was busy with other stuff - please find attached a proposal patch for the test - I don't know enough about the cmake testing infrastructure to say whether that was what you imagined, please let me know if I should change anything. Cheers, /Manuel On Thu, Jan 20, 2011 at 1:55 PM, Brad King brad.k...@kitware.com wrote: On 1/19/2011 9:07 PM, Manuel Klimek wrote: So the question is: do you want to run this test as part of the dashboard clients? Yes, we do. Please implement the test in C++ using the hints I sent previously. Thanks, -Brad -- Manuel Klimek (http://go/klimek) cmake-compile-command-test.patch Description: Binary data ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Tue, Jan 18, 2011 at 11:08 AM, David Cole david.c...@kitware.com wrote: On Tue, Jan 18, 2011 at 1:53 PM, Brad King brad.k...@kitware.com wrote: On 1/18/2011 1:50 PM, Manuel Klimek wrote: Since this is a big integration style test, is it also possible to integrate a python test? (the whole test would fit into a few lines of python then) We'd prefer not to depend on python, but certainly a prototype in python would be a good start. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers Brad's being polite when he says we'd prefer not to depend on python. CMake itself *will not* depend on python. And for any test that is expected to be run on all of our dashboard clients before it's accepted into CMake 'master', such a test cannot depend on python either. So the question is: do you want to run this test as part of the dashboard clients? I'm happy to comply with whatever decision you make, given the trade-off between maintainability of the test and dependency on python. Cheers, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Mon, Jan 17, 2011 at 8:52 AM, Brad King brad.k...@kitware.com wrote: On 01/14/2011 06:19 PM, Manuel Klimek wrote: Attached are 2 patches for caching the definesflags and introducing the options. Thanks. I'll look into testing next, any hints for what to look out for would be appreciated ;) I think the test needs to build one of CMake's current tests, such as COnly, with the new export option. Then it should parse the json file and try re-running all the recorded command lines. The easiest way to do this is probably to create a new executable that builds as part of the main CMake build process when the BUILD_TESTING option is on. It can link to CMakeLib and use helpers like cmSystemTools::RunSingleCommand to take care of re-running the children. You'll have to write a tiny json parser of course. Since this is a big integration style test, is it also possible to integrate a python test? (the whole test would fit into a few lines of python then) Cheers, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Tue, Jan 18, 2011 at 1:53 PM, Brad King brad.k...@kitware.com wrote: On 1/18/2011 1:50 PM, Manuel Klimek wrote: Since this is a big integration style test, is it also possible to integrate a python test? (the whole test would fit into a few lines of python then) We'd prefer not to depend on python, but certainly a prototype in python would be a good start. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers Brad's being polite when he says we'd prefer not to depend on python. CMake itself *will not* depend on python. And for any test that is expected to be run on all of our dashboard clients before it's accepted into CMake 'master', such a test cannot depend on python either. David ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 01/14/2011 01:28 PM, Manuel Klimek wrote: I'm fine with making it optional. Another option would be to make it hidden, as we want tools and not users to interact with it. Your call. Actually, with an option then the file name becomes moot: $ cmake ... -DCMAKE_EXPORT_COMPILE_RULES=compile_commands.json Okay. Does JSON support any kind of schema? Why would we need a schema? For documenation? Or do you plan to use meta-tools on it? Just curious. Pretty much every time I've ever written code that generates a .xml file someone eventually comes along asking for a schema. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Fri, Jan 14, 2011 at 10:53 AM, Brad King brad.k...@kitware.com wrote: On 01/14/2011 01:28 PM, Manuel Klimek wrote: I'm fine with making it optional. Another option would be to make it hidden, as we want tools and not users to interact with it. Your call. Actually, with an option then the file name becomes moot: $ cmake ... -DCMAKE_EXPORT_COMPILE_RULES=compile_commands.json Hm, I was actually going more for a bool option - I'd really like a file name that's easily recognizable by a tool with minimal user configuration. We can of course put in a second option with the file name with a good default. Cheers, /Manuel Okay. Does JSON support any kind of schema? Why would we need a schema? For documenation? Or do you plan to use meta-tools on it? Just curious. Pretty much every time I've ever written code that generates a .xml file someone eventually comes along asking for a schema. ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 01/14/2011 01:58 PM, Manuel Klimek wrote: Hm, I was actually going more for a bool option - I'd really like a file name that's easily recognizable by a tool with minimal user configuration. Sure. If you are trying to create a standard file name/format then you should choose a marketable name for it. That will make it recognizable, distinct, googleable, and not tied to CMake or any build system. It will also provide a meaningful name for the CMake option: $ cmake ... -DCMAKE_EXPORT_$coolname=1 -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 01/14/2011 05:46 PM, Manuel Klimek wrote:
I'm currently trying to figure out the best point where to add that
option...
In the cmMakefileTargetGenerator you can test for the option with
this-Makefile-IsOn(...)
Is there a good example I can base it on? So far I followed
CMAKE_BUILD_TYPE through the code base, which suggests that I need to
- find a .cmake file in modules in which I can put the option
- document it in cmDocumentVariables
Probably like CMAKE_VERBOSE_MAKEFILE which is another option to the
Makefile generator. See CMakeGenericSystem.cmake. Surround it by
if(${CMAKE_GENERATOR} MATCHES Make)
...
endif()
so that it only shows up for Makefile generators.
-Brad
___
cmake-developers mailing list
cmake-developers@cmake.org
http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On 01/12/2011 11:47 PM, Manuel Klimek wrote: Split up the patch - apparently I need ssh access to push the topic stuff, so I'm just attaching the patches. Thanks. That's pretty clean. In the final version of this topic please add one more intermediate commit that optimizes GetFlags and GetDefines to cache their result for each language. Otherwise they get invoked once for the target (as in current CMake) plus once for each source file (after the second patch). About the name cxx_commands.json of the file: I'd like to hide this away from the casual user by putting it in the CMakeFiles subdirectory. The commands can be for compilation of any language so it should be called something like CMakeFiles/compile_commands.json instead. About the content of the json file: I suggest that each command also include a language field indicating C, C++, or Fortran. That will help tools know how to treat the file named by the file field. One might also include a target field indicating the name of the logical target to which it belongs. I don't know whether this is useful in your case or not. -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
On Thu, Jan 13, 2011 at 1:30 PM, Brad King brad.k...@www.kitware.com wrote: On 01/12/2011 11:47 PM, Manuel Klimek wrote: Split up the patch - apparently I need ssh access to push the topic stuff, so I'm just attaching the patches. Thanks. That's pretty clean. In the final version of this topic please add one more intermediate commit that optimizes GetFlags and GetDefines to cache their result for each language. Otherwise they get invoked once for the target (as in current CMake) plus once for each source file (after the second patch). Will do. About the name cxx_commands.json of the file: I'd like to hide this away from the casual user by putting it in the CMakeFiles subdirectory. The commands can be for compilation of any language so it should be called something like CMakeFiles/compile_commands.json instead. I think that finding this file in the same spot relative to the base build directory independently of the build system used will make it a lot easier for tool-smiths / users to actually make use of it. If you want to hide it away, I'd vote for a more build system independent name than CMakeFiles. The commands are currently only for c/c++ compilation, but I'm fine with compile_commands to keep it flexible. About the content of the json file: I suggest that each command also include a language field indicating C, C++, or Fortran. That will help tools know how to treat the file named by the file field. One might also include a target field indicating the name of the logical target to which it belongs. I don't know whether this is useful in your case or not. I'd rather keep it focused for now, and add additional things later if you're ok with that. Cheers, /Manuel ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
Hi Manuel, I think the goals of your work are important. I like the general idea. One sure way to give such tools the same view of the code as the compiler is to ask the build system to provide the information. I'm a CMake developer but I also wrote gccxml. The latter is a code analysis tool that simulates the preprocessing performed by a target compiler to get the same view of the code. We use it to convert C++ header files into a .xml representation of the interface for easy loading by interpreted language wrapper generators. In order to run gccxml in real projects we need to write CMake code that uses add_custom_command to create build-time rules. Passing the include path and preprocessor definitions requires custom code to add them to the command line, and there is no guarantee that they stay in sync with what the real compiler gets. This is difficult to maintain. One distinction between gccxml and tools like you propose is that gccxml has to run during the build to help generate code for the compiler rather than running after the build to analyze the code. However, the difficulty with running it to match the compiler demonstrates why build system awareness is helpful. On 1/11/2011 5:10 PM, Manuel Klimek wrote: I thought about that, too. On the other hand, we might also want to make that information available from other generators where it makes sense - in my mind it is somewhat orthogonal to the generator: - for generators that are already very tightly IDE integrated (like VS) we probably don't need to output this information, as the IDE already exposes this information in well-defined ways that tool users for those IDEs would use - for generators that are not tightly coupled to the IDE or where the IDE doesn't make the build information easily accessible (like Eclipse), we actually want to enable developers to make the output default in their cmake build setup, so they can use command line tools or hand-made macros in their editor/IDE. CMake has 2 categories of main generators: IDE and Makefile. The Xcode and VS generators are IDE, the rest are Makefile. Other generators like KDevelop are actually Makefile generators plus project files that just list source files but not build rules. The IDE generators (Xcode and VS) do not actually generate the final compile lines. Therefore the proposed functionality is possible only with Makefile generators. However, since there is no toolchain not supported by Makefile generators that should be sufficient. No, I'm not planning to output the link commands - this is purely for code understanding, not rebuilding the code - while thinking about how to enable fast rebuilds by adding this information might have some valid use case, it is currently completely out of scope as far as I'm concerned. My understanding is that clang is based on llvm which is a set of modular components for all parts of the toolchain. It is conceivable that tools in the future may be interested in the link line to do whole-program analysis. However, I think starting off with just the compile information is sufficient. - you use your normal cmake+(your build system) workflow for building - you use tools that run over source files and use the little c++ compile database to be able to understand the c++ code correctly As a user, I want to be able to do both without having to run cmake in between. This might well be hidden behind a cmake configuration option if you think it affects the standard generation too much. I need to review your patch in more detail before commenting on its effects on standard generation. If it is just a matter of writing one extra file per logical target (created by add_executable, add_library, or add_custom_target), then it shouldn't be too bad. How do you propose to cover the new functionality in our test suite? Currently your patch is one big change, but it looks like it factors some existing code out into smaller pieces. Do you mind splitting it into multiple Git commits in a topic branch? Ideally the first commit(s) would just do the refactoring without changing behavior and the later commit(s) would add the new functionality. This level of granularity in the commits will make your change easier to review. Thanks! -Brad ___ cmake-developers mailing list cmake-developers@cmake.org http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Generating information for C++ tools in cmake (patch)
Hi Brad, On Wed, Jan 12, 2011 at 8:25 AM, Brad King brad.k...@kitware.com wrote: Hi Manuel, I think the goals of your work are important. I like the general idea. One sure way to give such tools the same view of the code as the compiler is to ask the build system to provide the information. I'm a CMake developer but I also wrote gccxml. The latter is a code analysis tool that simulates the preprocessing performed by a target compiler to get the same view of the code. We use it to convert C++ header files into a .xml representation of the interface for easy loading by interpreted language wrapper generators. In order to run gccxml in real projects we need to write CMake code that uses add_custom_command to create build-time rules. Passing the include path and preprocessor definitions requires custom code to add them to the command line, and there is no guarantee that they stay in sync with what the real compiler gets. This is difficult to maintain. One distinction between gccxml and tools like you propose is that gccxml has to run during the build to help generate code for the compiler rather than running after the build to analyze the code. However, the difficulty with running it to match the compiler demonstrates why build system awareness is helpful. Exporting the information in a defined way into the script language would be an option, too. If we have the information for example as a readable property for source files, writing the code that exports the data in any format in the cmake file is trivial - on the other hand I'd really like to create some form of well-defined standard format that tools can rely on finding. One other idea that looks like it would help especially your case would be to add an extra target command called cxx_tool or something that serves the purpose of running tools that need the c++ command line. I'd be also happy to spend some cycles on helping to implement that for the makefile case, as we're planning on tools like include-what-you-use (http://llvm.org/devmtg/2010-11/Silverstein-IncludeWhatYouUse.pdf) that you might want to run during a build, too. On 1/11/2011 5:10 PM, Manuel Klimek wrote: I thought about that, too. On the other hand, we might also want to make that information available from other generators where it makes sense - in my mind it is somewhat orthogonal to the generator: - for generators that are already very tightly IDE integrated (like VS) we probably don't need to output this information, as the IDE already exposes this information in well-defined ways that tool users for those IDEs would use - for generators that are not tightly coupled to the IDE or where the IDE doesn't make the build information easily accessible (like Eclipse), we actually want to enable developers to make the output default in their cmake build setup, so they can use command line tools or hand-made macros in their editor/IDE. CMake has 2 categories of main generators: IDE and Makefile. The Xcode and VS generators are IDE, the rest are Makefile. Other generators like KDevelop are actually Makefile generators plus project files that just list source files but not build rules. The IDE generators (Xcode and VS) do not actually generate the final compile lines. Therefore the proposed functionality is possible only with Makefile generators. However, since there is no toolchain not supported by Makefile generators that should be sufficient. Thanks for explaining. No, I'm not planning to output the link commands - this is purely for code understanding, not rebuilding the code - while thinking about how to enable fast rebuilds by adding this information might have some valid use case, it is currently completely out of scope as far as I'm concerned. My understanding is that clang is based on llvm which is a set of modular components for all parts of the toolchain. It is conceivable that tools in the future may be interested in the link line to do whole-program analysis. However, I think starting off with just the compile information is sufficient. Yes, I agree that it is very conceivable that we'll want that in the future - one reason I propose a JSON format is that it is both easy to parse and easy to extend. - you use your normal cmake+(your build system) workflow for building - you use tools that run over source files and use the little c++ compile database to be able to understand the c++ code correctly As a user, I want to be able to do both without having to run cmake in between. This might well be hidden behind a cmake configuration option if you think it affects the standard generation too much. I need to review your patch in more detail before commenting on its effects on standard generation. If it is just a matter of writing one extra file per logical target (created by add_executable, add_library, or add_custom_target), then it shouldn't be too bad. It is actually a matter of writing one
[cmake-developers] Generating information for C++ tools in cmake (patch)
Hi,
we're working on C++ tools based on clang to get the power of
automated refactoring and analysis that Java has known for years to
C++ developers.
With the attached patch I include a proposal for how we could export
the compile command line for C/C++ files from cmake for Unix makefile
generators:
The idea is to output a small JSON database with all C/C++ files in
the project, which contains
- the file's path
- the working directory in which the compile runs
- the compile command
Background:
While C++ refactoring and analysis tools are not compilers, and thus
don't run as part of the build system, they need the exact information
of a build in order to be able to correctly understand the C++ code of
the project. The natural candidate to export that information is the
build system. We chose cmake to provide a first implementation, as
cmake is the current open source standard build system for large scale
C++ projects (like clang itself), and as such it is in the best
position to define a standard way to make compile commands accessible
for tools in the open source world.
Example tools:
- running clang -fsyntax-only over source code from an editor to get
fast syntax checks
- running match/replace tools over C++ code (a project we're currently
working on)
Any feedback would be highly appreciated. If you think the way this is
implemented is a dumb idea (which it probably is), please tell me - I
don't have a lot of experience in the cmake architecture. Also, if you
object to the idea of exporting this information in general, please
let me know.
Thanks a lot,
/Manuel
diff --git a/Source/cmGlobalUnixMakefileGenerator3.cxx b/Source/cmGlobalUnixMakefileGenerator3.cxx
index d9a341c..92f87c9 100644
--- a/Source/cmGlobalUnixMakefileGenerator3.cxx
+++ b/Source/cmGlobalUnixMakefileGenerator3.cxx
@@ -31,6 +31,7 @@ cmGlobalUnixMakefileGenerator3::cmGlobalUnixMakefileGenerator3()
#else
this-UseLinkScript = true;
#endif
+ this-CommandDatabase = NULL;
}
void cmGlobalUnixMakefileGenerator3
@@ -139,6 +140,17 @@ void cmGlobalUnixMakefileGenerator3
}
//
+std::string EscapeJSON(const std::string s) {
+ std::string result;
+ for (int i = 0; i s.size(); ++i) {
+if (s[i] == '' || s[i] == '\\') {
+ result += '\\';
+}
+result += s[i];
+ }
+ return result;
+}
+
void cmGlobalUnixMakefileGenerator3::Generate()
{
// first do superclass method
@@ -189,6 +201,35 @@ void cmGlobalUnixMakefileGenerator3::Generate()
// write the main makefile
this-WriteMainMakefile2();
this-WriteMainCMakefile();
+
+ if (this-CommandDatabase != NULL) {
+*this-CommandDatabase std::endl ];
+delete this-CommandDatabase;
+this-CommandDatabase = NULL;
+ }
+}
+
+void cmGlobalUnixMakefileGenerator3::AddCXXCompileCommand(
+const std::string sourceFile, const std::string workingDirectory,
+const std::string compileCommand) {
+ if (this-CommandDatabase == NULL)
+{
+std::string commandDatabaseName =
+ std::string(this-GetCMakeInstance()-GetHomeOutputDirectory())
+ + /cxx_commands.json;
+this-CommandDatabase =
+ new cmGeneratedFileStream(commandDatabaseName.c_str());
+*this-CommandDatabase [ std::endl;
+} else {
+*this-CommandDatabase , std::endl;
+}
+ *this-CommandDatabase { std::endl
+ \directory\: \ EscapeJSON(workingDirectory) \,
+ std::endl
+ \command\: \ EscapeJSON(compileCommand) \,
+ std::endl
+ \file\: \ EscapeJSON(sourceFile) \
+ std::endl };
}
void cmGlobalUnixMakefileGenerator3::WriteMainMakefile2()
diff --git a/Source/cmGlobalUnixMakefileGenerator3.h b/Source/cmGlobalUnixMakefileGenerator3.h
index cdc9460..a152e01 100644
--- a/Source/cmGlobalUnixMakefileGenerator3.h
+++ b/Source/cmGlobalUnixMakefileGenerator3.h
@@ -112,6 +112,10 @@ public:
/** Record per-target progress information. */
void RecordTargetProgress(cmMakefileTargetGenerator* tg);
+ void AddCXXCompileCommand(const std::string sourceFile,
+const std::string workingDirectory,
+const std::string compileCommand);
+
protected:
void WriteMainMakefile2();
void WriteMainCMakefile();
@@ -178,6 +182,8 @@ protected:
size_t CountProgressMarksInTarget(cmTarget* target,
std::setcmTarget* emitted);
size_t CountProgressMarksInAll(cmLocalUnixMakefileGenerator3* lg);
+
+ cmGeneratedFileStream *CommandDatabase;
};
#endif
diff --git a/Source/cmMakefileTargetGenerator.cxx b/Source/cmMakefileTargetGenerator.cxx
index 9dcd8f1..c14b789 100644
--- a/Source/cmMakefileTargetGenerator.cxx
+++ b/Source/cmMakefileTargetGenerator.cxx
@@ -249,6 +249,62 @@ void cmMakefileTargetGenerator::WriteCommonCodeRules()
}
//
+std::string cmMakefileTargetGenerator::GetFlags(const
