Re: [cmake-developers] Profile Cmake scripts

[Bartosz Kosiorek]

Hi Ben.

2016-01-04 17:39 GMT+01:00 Ben Boeckel :

> The
> big remaining problem is passing char* as an argument where functions do
> std::string(arg) right away. I fixed all of those which did explicit
> std::string conversions (via assignment or otherwise), but those which
> are conditional should get std::string overloads to avoid a
> malloc/copy/free penalty. There is a *lot* of work here though. The
> branch I had been working on (which is now woefully out-of-date) is 100
> commits long.
>

That's great news.
What is the branch name/link to these improvement?
Is it possible to push these improvements partially?

Maybe CMake community could continue working on that improvement?
It will be great to create ticket, in which propose solution will be
described.

BTW: Do you know why the Xcode and MS Visual Studio is slower than CMake
one ?

Here is the benchmarks for my internal project:

Xcode generation:
   real 6m31.733s
   user 4m51.862s
   sys 0m20.268s

Make generation:
   real 4m45.089s
   user 2m56.117s
   sys 0m17.481s

Ninja generation:
  real 2m48.585s
  user 2m30.712s
  sys 0m6.313s





>
> --Ben
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake-developers
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Profile Cmake scripts

[Ben Boeckel]

On Tue, Jan 05, 2016 at 09:18:49 +0100, Bartosz Kosiorek wrote:
> That's great news.
> What is the branch name/link to these improvement?
> Is it possible to push these improvements partially?

Because the changes are so pervasive, I'd like for the branch to be
bisectable, so every commit should pass the test suite on at least the
common platforms (make, ninja, visual studio, xcode).

https://github.com/mathstuf/cmake/tree/dev/string-overloads

> Maybe CMake community could continue working on that improvement?

I have no problem with anyone taking over the branch as long as the
branch ends up bisectable. If anyone wants to take it over and put the
branch together, I can run the test suite.

> It will be great to create ticket, in which propose solution will be
> described.
> 
> BTW: Do you know why the Xcode and MS Visual Studio is slower than CMake
> one ?

As Petr said, it is because these generators generate a build for each
configuration. You can help this by removing configurations you don't
care about from CMAKE_CONFIGURATION_TYPES (like, say, MinSizeRel) to
help reduce the time.

There are also probably some places in the genex evaluator where
computed values could be cached to improve times (e.g., make a cache of
the transitive closure for each target's include directories, link
libraries, etc. when evaluating $ and
friends). Alternatively, cmTarget could cache them (but that may not be
as straight forward since cmTarget is editable while genex evaluation
occurs when everything is read-only). Not sure how much it would help
the multi-config generators since they already memoize
configuration-independent evaluations.

--Ben
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Profile Cmake scripts

[Daniel Pfeifer]

On Tue, Jan 5, 2016 at 9:18 AM, Bartosz Kosiorek  wrote:
>
> Hi Ben.
>
> 2016-01-04 17:39 GMT+01:00 Ben Boeckel :
>>
>> The
>> big remaining problem is passing char* as an argument where functions do
>> std::string(arg) right away. I fixed all of those which did explicit
>> std::string conversions (via assignment or otherwise), but those which
>> are conditional should get std::string overloads to avoid a
>> malloc/copy/free penalty. There is a *lot* of work here though. The
>> branch I had been working on (which is now woefully out-of-date) is 100
>> commits long.

This improvement should be semi-automated with clang-tidy. We have
many functions that take `const char*` and are called by passing a
`string.c_str()`. The function signature should be changed to `const
std::string&` manually, but changing all the call sites should be done
automatically (ideally by kwrobot, periodically, on multiple
platforms).

Here is how to run clang-tidy on projects built with CMake:

$ cd 
$ cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON 
$ run-clang-tidy.py -p  -fix
-checks='-*,readability-redundant-string-cstr' /Source/

The run-clang-tidy.py script is here:
https://llvm.org/svn/llvm-project/clang-tools-extra/trunk/clang-tidy/tool/run-clang-tidy.py
The `-fix` argument tells clang-tidy to fix the violations inplace.
Without this argument the diagnostics are just printed.

I wanted to demonstrage the ability of the `redundant-string-cstr`
checker, but apparently there are no violations found.
I have attached a patch that was generated by the
`container-size-empty` checker.
Please don't apply it. Such changes should be performed by kwrobot.

You can output all possible checks with:

$ clang-tidy -p  -checks='*' -list-checks 

> That's great news.
> What is the branch name/link to these improvement?
> Is it possible to push these improvements partially?
>
> Maybe CMake community could continue working on that improvement?
> It will be great to create ticket, in which propose solution will be
> described.
>
> BTW: Do you know why the Xcode and MS Visual Studio is slower than CMake one
> ?
>
> Here is the benchmarks for my internal project:
>
> Xcode generation:
>real 6m31.733s
>user 4m51.862s
>sys 0m20.268s
>
> Make generation:
>real 4m45.089s
>user 2m56.117s
>sys 0m17.481s
>
> Ninja generation:
>   real 2m48.585s
>   user 2m30.712s
>   sys 0m6.313s

Generators for Xcode and Visual Studio have to generate more files.
From 46a5639a27bab0b227b457e1224c702dd143af33 Mon Sep 17 00:00:00 2001
From: Daniel Pfeifer 
Date: Tue, 5 Jan 2016 13:28:16 +0100
Subject: [PATCH] demonstrate clang-tidy

---
 Source/CTest/cmCTestRunTest.cxx | 4 ++--
 Source/CTest/cmCTestTestHandler.cxx | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/Source/CTest/cmCTestRunTest.cxx b/Source/CTest/cmCTestRunTest.cxx
index d108592..87e9f62 100644
--- a/Source/CTest/cmCTestRunTest.cxx
+++ b/Source/CTest/cmCTestRunTest.cxx
@@ -207,7 +207,7 @@ bool cmCTestRunTest::EndTest(size_t completed, size_t total, bool started)
 {
 bool success =
   !forceFail &&  (retVal == 0 ||
-  this->TestProperties->RequiredRegularExpressions.size());
+  !this->TestProperties->RequiredRegularExpressions.empty());
 if(this->TestProperties->SkipReturnCode >= 0
   && this->TestProperties->SkipReturnCode == retVal)
   {
@@ -584,7 +584,7 @@ void cmCTestRunTest::ComputeArguments()
  << std::endl);
 
   // Print any test-specific env vars in verbose mode
-  if (this->TestProperties->Environment.size())
+  if (!this->TestProperties->Environment.empty())
 {
 cmCTestLog(this->CTest, HANDLER_VERBOSE_OUTPUT, this->Index << ": "
<< "Environment variables: " << std::endl);
diff --git a/Source/CTest/cmCTestTestHandler.cxx b/Source/CTest/cmCTestTestHandler.cxx
index a8f983f..df91ac5 100644
--- a/Source/CTest/cmCTestTestHandler.cxx
+++ b/Source/CTest/cmCTestTestHandler.cxx
@@ -1318,7 +1318,7 @@ void cmCTestTestHandler::AttachFiles(cmXMLWriter& xml,
  cmCTestTestResult* result)
 {
   if(result->Status != cmCTestTestHandler::COMPLETED
- && result->Properties->AttachOnFail.size())
+ && !result->Properties->AttachOnFail.empty())
 {
 result->Properties->AttachedFiles.insert(
   result->Properties->AttachedFiles.end(),
-- 
2.6.4

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to 

Re: [cmake-developers] Profile Cmake scripts

[Petr Kmoch]

On Tue, Jan 5, 2016 at 1:33 PM, Daniel Pfeifer 
wrote:

>
> Generators for Xcode and Visual Studio have to generate more files.
>

... and they have to process all configurations (Debug, Release,
MinSizeRel, RelWithDbgInfo) during the generation stage, instead of just
one being processed for single-config generators like Makefiles or Ninja.

Petr
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Profile Cmake scripts

[Ben Boeckel]

On Sun, Dec 27, 2015 at 22:04:14 +0100, Dimitar Yordanov wrote:
> I agree with you. Running valgrind directly on the cmake binary
> provides useful information: I can see which internal cmake functions
> are used the most and consume most of the time.
> 
> Nevertheless, I think it would be useful to have a higher level
> overview. E.g. to see if there are some issues with the scripts
> themselves that I use in my project ...

I had patches which did this, but it doesn't really help much. At least
I didn't think it did. The problems are mainly in CMake itself (regular
expressions, path parsing, etc.). I think the next big improvement would
be caching variable values which are parsed as numbers, on/off, and
paths as a more structured state so that if you have a path, path
operations don't have to keep searching for directory separators since
it is already in a std::vector structure.

> > Usually you see mostly string handling related functions.
> 
> malloc and free are on the top of what I see for a random project used
> mostly by std::string. Maybe we can optimize something here too?

I did a lot of profiling of CMake two years ago. I fixed the low-hanging
fruit (rewriting the CMakeLists.txt parser/variable expander (CMP0053 if
you aren't using it), fixing other mini-parsers (escape routines, the
genex parser) to chunk rather than do char-by-char iteration, etc.). The
big remaining problem is passing char* as an argument where functions do
std::string(arg) right away. I fixed all of those which did explicit
std::string conversions (via assignment or otherwise), but those which
are conditional should get std::string overloads to avoid a
malloc/copy/free penalty. There is a *lot* of work here though. The
branch I had been working on (which is now woefully out-of-date) is 100
commits long.

--Ben
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Profile Cmake scripts

[Alexander Neundorf]

On Sunday, December 27, 2015 18:40:27 Dimitar Yordanov wrote:
> Hi all,
> 
> I was searching for a way to profile CMake scripts in order to find
> bottlenecks and possibilities to improve performance. I found out that
> someone already invested time on that [1] providing a minimal 
solution. The
> idea behind it is to use the cmake "--trace" option and to output a time
> stamp and a stack depth with each executed command. The trace log 
is
> afterwards used by a Python script, which transforms it into a more
> readable format.

I actually simply used valgrind/callgrind and got useful results from that.
Usually you see mostly string handling related functions.

Alex

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Profile Cmake scripts

[Dimitar Yordanov]

2015-12-27 21:05 GMT+01:00 Alexander Neundorf :
>
> On Sunday, December 27, 2015 18:40:27 Dimitar Yordanov wrote:
>
> > Hi all,
>
> >
>
> > I was searching for a way to profile CMake scripts in order to find
>
> > bottlenecks and possibilities to improve performance. I found out that
>
> > someone already invested time on that [1] providing a minimal solution. The
>
> > idea behind it is to use the cmake "--trace" option and to output a time
>
> > stamp and a stack depth with each executed command. The trace log is
>
> > afterwards used by a Python script, which transforms it into a more
>
> > readable format.
>
>
>
> I actually simply used valgrind/callgrind and got useful results from that.

I agree with you. Running valgrind directly on the cmake binary
provides useful information: I can see which internal cmake functions
are used the most and consume most of the time.

Nevertheless, I think it would be useful to have a higher level
overview. E.g. to see if there are some issues with the scripts
themselves that I use in my project ...

> Usually you see mostly string handling related functions.

malloc and free are on the top of what I see for a random project used
mostly by std::string. Maybe we can optimize something here too?

Regards,
Dimitar
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers