[cmake-developers] [CMake 0014342]: CMAKE_SYSTEM_PROCESS behaviour changed
The following issue has been SUBMITTED. == http://public.kitware.com/Bug/view.php?id=14342 == Reported By:Craig Scott Assigned To: == Project:CMake Issue ID: 14342 Category: CMake Reproducibility:always Severity: major Priority: normal Status: new == Date Submitted: 2013-08-06 23:11 EDT Last Modified: 2013-08-06 23:11 EDT == Summary:CMAKE_SYSTEM_PROCESS behaviour changed Description: The behaviour of the value of CMAKE_SYSTEM_PROCESSOR appears to have changed somewhere between 2.8.10.1 and 2.8.11.2. I build 32-bit apps on a 64-bit Windows 7 system and I use VS2008 (express version, no 64-bit compiler). With CMake 2.8.10.1, CMAKE_SYSTEM_PROCESSOR is reported to have the value x86, but with 2.8.11.2, now it has the value AMD64. I verified this with both versions of CMake and testing a trivial CMake project that simply prints the value of the variable. According to the CMake documentation, on the one hand, the value should have always been AMD64 since the documentation says it just pulls the value from the PROCESSOR_ARCHITECTURE environment variable. On the other hand, this conflicts with the statement that it should reflect the target architecture. I think x86 would be the correct value here rather than simply reporting whatever PROCESSOR_ARCHITECTURE says, since that environment variable is not going to always match the target architecture. == Issue History Date ModifiedUsername FieldChange == 2013-08-06 23:11 Craig ScottNew Issue == -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
On Tue, Aug 6, 2013 at 10:19 PM, Brad King wrote: > On 08/06/2013 01:30 PM, Brad King wrote: > > IMO it is not worth the effort to establish this test. > > I've applied a patch based on yours to implement the > optimization and merged to 'next' for testing tonight: > > http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=2268c41a > > Thank you very much for your help. I guess this topic is closed now. Sorry for not answering earlier. I was away from keyboard (time shift). I will delete my branches tomorrow. -Nico -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
On 08/06/2013 01:30 PM, Brad King wrote: > IMO it is not worth the effort to establish this test. I've applied a patch based on yours to implement the optimization and merged to 'next' for testing tonight: http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=2268c41a -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
On 08/06/2013 12:44 PM, Brad King wrote:
> On 08/06/2013 12:18 PM, Nicolas Desprès wrote:
>> The test suite passes on my computer. I have force-pushed again.
>
> Great.
I had to modify the test like this:
-DEPENDS bottle_neck
+DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/bottle_neck
in order to ensure the optimization is used for the test to
finish within the timeout.
Also the test does not work well on Windows because the
filesystem is very slow to deal with a large number of
files and directories. Even removing creation of the
input_${i} files the 1000 output-target build makefiles
take forever to write.
IMO it is not worth the effort to establish this test.
We have no performance tests elsewhere and it is unlikely
this will regress. The way the lookup is done with your
patches is the way it should have been done originally.
-Brad
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
On 08/06/2013 12:18 PM, Nicolas Desprès wrote: > The test suite passes on my computer. I have force-pushed again. Great. In this hunk: + // If the queried path is not absolute we use the backward compatible + // version. The search algorithm is linear. + if (cname[0] != '/') Use cmSystemTools::FileIsFullPath to detect a full path in a way that works on Windows too. > However I had one problem with BootstrapTest when trying to use > cmsys::hash_map During bootstrap we can just use normal std::map. Squash the patch below into your topic/use-cmsys-hash_map (and then into the original topic). -Brad diff --git a/Source/cmMakefile.h b/Source/cmMakefile.h index e51d9c6..f00f47b 100644 --- a/Source/cmMakefile.h +++ b/Source/cmMakefile.h @@ -29,7 +29,9 @@ #include #include -#include +#if defined(CMAKE_BUILD_WITH_CMAKE) +# include +#endif class cmFunctionBlocker; class cmCommand; @@ -1049,7 +1051,11 @@ private: cmSourceFile *LinearGetSourceFileWithOutput(const char *cname); // A map for fast output to input look up. +#if defined(CMAKE_BUILD_WITH_CMAKE) typedef cmsys::hash_map OutputToSourceMap; +#else + typedef std::map OutputToSourceMap; +#endif OutputToSourceMap OutputToSource; void UpdateOutputToSourceMap(const std::vector& outputs, diff --git a/bootstrap b/bootstrap index 9e6bfad..afb66e5 100755 --- a/bootstrap +++ b/bootstrap @@ -304,8 +304,6 @@ KWSYS_CXX_SOURCES="\ KWSYS_FILES="\ auto_ptr.hxx \ Directory.hxx \ - hash_map.hxx \ - hashtable.hxx \ Glob.hxx \ Process.h \ RegularExpression.hxx \ -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
On Tue, Aug 6, 2013 at 4:31 PM, Brad King wrote:
> On 08/06/2013 09:40 AM, Nicolas Desprès wrote:
> > I have force-pushed all the fixes you've asked for:
>
> Great, thanks!
>
> > * I taught cmMakefile copy-ctor
>
> Good.
>
> > * I re-write the test so that it is part of CMakeOnly sub-test suite. I
> also make it more robust to machine load by increasing the number of
> targets. This way the performance gap between the logarithmic version and
> the linear version is bigger. On my machine the logarithmic version run in
> 60sec
> > whereas the linear version had not finished after 10min. The timeout is
> at 90sec now. When running the whole test suite in parallel with 8 jobs it
> passes.
>
> Okay. Timing-dependent tests have been frequently sporadic on
> our test infrastructure because the machines are often busy
> with other projects' tests and there is a wide variety of
> machine speeds available. We'll see how it goes.
>
Sure.
>
> > * I added a test covering the issue I got while testing the previous
> buggy version of the optimized algorithm. This is a very weird bug. Master
> does not suffer from it, neither this topic branch. Basically the previous
> version had problem with empty dependency and it was not covered by the test
> > suite. I think there is something else involved in this test case
> because the test file named "0" is important too. At least the test is here
> now and we could investigate that later if it fails again.
>
> It looks like
>
> + add_test_macro(EmptyDepAndZeroOutput
> +${CMAKE_CMAKE_COMMAND} -P check.cmake)
>
> appears twice, once in each test addition commit.
>
Oups. Squashing error. Fixed now.
>
> While hunting down all the call sites for GetSourceFileWithOutput
> I found two that are not needed and removed them:
>
> http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=eccb39d7
>
> Please rebase your topic on that.
>
> I still can't convince myself that the comparison function is
> correct. Also as Steve suggested it will be faster to use a
> standard ordering function. That should work if we always
> use it with full paths. The expected use cases always use
> full paths and it is only for backward compatibility that we
> support the path suffix magic. Therefore I think the best
> approach is to optimize the common (full-path) case with a
> direct lookup, perhaps even with "cmsys/hash_map.hxx". Only
> when the input path is relative should we fall back to the
> linear suffix search for compatibility.
Good idea! I should have done that in the first place it would have save me
a lot of effort understanding the old behavior.
> The test suite passes on my computer. I have force-pushed again.
However I had one problem with BootstrapTest when trying to use
cmsys::hash_map. I pushed by work so far on this topic in another branch (
https://github.com/nicolasdespres/CMake/commits/topic/use-cmsys-hash_map):
https://github.com/nicolasdespres/CMake/commit/1437d51a87b5a5a131b3829210710734266d29cc
I did not succeed to teach the bootstrap script how to include cstddef.
Cheers,
--
Nicolas Desprès
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
On 08/06/2013 09:40 AM, Nicolas Desprès wrote:
> I have force-pushed all the fixes you've asked for:
Great, thanks!
> * I taught cmMakefile copy-ctor
Good.
> * I re-write the test so that it is part of CMakeOnly sub-test suite. I also
> make it more robust to machine load by increasing the number of targets. This
> way the performance gap between the logarithmic version and the linear
> version is bigger. On my machine the logarithmic version run in 60sec
> whereas the linear version had not finished after 10min. The timeout is at
> 90sec now. When running the whole test suite in parallel with 8 jobs it
> passes.
Okay. Timing-dependent tests have been frequently sporadic on
our test infrastructure because the machines are often busy
with other projects' tests and there is a wide variety of
machine speeds available. We'll see how it goes.
> * I added a test covering the issue I got while testing the previous buggy
> version of the optimized algorithm. This is a very weird bug. Master does not
> suffer from it, neither this topic branch. Basically the previous version had
> problem with empty dependency and it was not covered by the test
> suite. I think there is something else involved in this test case because the
> test file named "0" is important too. At least the test is here now and we
> could investigate that later if it fails again.
It looks like
+ add_test_macro(EmptyDepAndZeroOutput
+${CMAKE_CMAKE_COMMAND} -P check.cmake)
appears twice, once in each test addition commit.
While hunting down all the call sites for GetSourceFileWithOutput
I found two that are not needed and removed them:
http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=eccb39d7
Please rebase your topic on that.
I still can't convince myself that the comparison function is
correct. Also as Steve suggested it will be faster to use a
standard ordering function. That should work if we always
use it with full paths. The expected use cases always use
full paths and it is only for backward compatibility that we
support the path suffix magic. Therefore I think the best
approach is to optimize the common (full-path) case with a
direct lookup, perhaps even with "cmsys/hash_map.hxx". Only
when the input path is relative should we fall back to the
linear suffix search for compatibility.
Thanks,
-Brad
--
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
[cmake-developers] Module.GenerateExportHeader test is sloooow
Anyone know why Module.GenerateExportHeader is so slow: http://open.cdash.org/testSummary.php?project=1&name=Module.GenerateExportHeader&date=2013-08-06 On the slowest build it takes 14 minutes to run. The kwsys test which builds a whole library and does a dashboard submission is faster... -Bill -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] CMake is slow for project with hundred target and one command with large number of dependencies
Hi Brad, I have force-pushed all the fixes you've asked for: * I taught cmMakefile copy-ctor * I re-write the test so that it is part of CMakeOnly sub-test suite. I also make it more robust to machine load by increasing the number of targets. This way the performance gap between the logarithmic version and the linear version is bigger. On my machine the logarithmic version run in 60sec whereas the linear version had not finished after 10min. The timeout is at 90sec now. When running the whole test suite in parallel with 8 jobs it passes. * I added a test covering the issue I got while testing the previous buggy version of the optimized algorithm. This is a very weird bug. Master does not suffer from it, neither this topic branch. Basically the previous version had problem with empty dependency and it was not covered by the test suite. I think there is something else involved in this test case because the test file named "0" is important too. At least the test is here now and we could investigate that later if it fails again. Let me know what do you think. Thanks, -Nico On Mon, Aug 5, 2013 at 10:50 PM, Brad King wrote: > On 08/05/2013 04:37 PM, Nicolas Desprès wrote: > > * Why is special logic in Tests/CMakeLists.txt needed to add > > the test case? > > > > Since it is a performance test on configuration phase only, I need a > > specific timeout value and to run cmake only > > Take a look at the Tests/RunCMake and Tests/CMakeOnly groups of > tests. They are for running CMake without building. > > -Brad > -- Nicolas Desprès -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Suspicious Clang versions
On 08/05/2013 03:45 PM, Brad King wrote: > One of the main purposes of the compiler is to load platform > modules like "Platform/--" and "Compiler/-". > Most of the information in these files will be identical across > the upstream and vendor-specific Clang distributions so having a > separate compiler id may lead to confusion and duplication. > > Another approach is to introduce a CMAKE__COMPILER_VENDOR > variable to name the vendor (or _VARIANT?). On second thought adding yet another dimension to the compiler identification is just more complexity. We can use a separate compiler for each variant and simply have the platform information modules for each vendor include the upstream one. -Brad -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
Re: [cmake-developers] Targets for FindGTK2.cmake
Hello Philip, Thanks for your comments. On 06/08/13 05:43, Philip Lowman wrote: > I'm not sure if there is a way to develop automated unit tests and mark > them to not run by default. That would be useful for regression testing > of modules even if it the tests had to be run manually. I added a commit introducing a couple of unit tests that should run only if GTK and/or GTKMM are available on the system. Regards, Daniele -- Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://public.kitware.com/cgi-bin/mailman/listinfo/cmake-developers
