Re: [cmake-developers] Proposal to teach cmDepends to only take dependencies from the source tree

[Attila Krasznahorkay]

Dear All,

I was hoping to get some feedback from the developers on which direction to 
take this in. As I wrote before, I'm happy to help. But I don't want to spend 
time on an implementation that people will not accept in the end. I'd like us 
to first agree on how the code should behave after the addition of this new 
feature.

Cheers,
  Attila

> On 09 Dec 2015, at 15:38, Ben Boeckel  wrote:
> 
> On Wed, Dec 09, 2015 at 09:19:45 +0100, Attila Krasznahorkay wrote:
>> I myself am on the side of simplicity. Using regular expressions would
>> allow us to do what I want. But if >95% of the people using this new
>> possibility would use it exactly like I plan to, then implementing it
>> in a more complicated way than necessary would just be a bad design
>> choice.
> 
> Do you have performance numbers for the change in your build between
> using include_directories (or target_include_directories), with and
> without SYSTEM, each with and without this change?
> 
> --Ben

-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Proposal to teach cmDepends to only take dependencies from the source tree

[Ben Boeckel]

On Wed, Dec 09, 2015 at 09:19:45 +0100, Attila Krasznahorkay wrote:
> I myself am on the side of simplicity. Using regular expressions would
> allow us to do what I want. But if >95% of the people using this new
> possibility would use it exactly like I plan to, then implementing it
> in a more complicated way than necessary would just be a bad design
> choice.

Do you have performance numbers for the change in your build between
using include_directories (or target_include_directories), with and
without SYSTEM, each with and without this change?

--Ben
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

[cmake-developers] Proposal to teach cmDepends to only take dependencies from the source tree

[Attila Krasznahorkay]

Dear All,

In our software infrastructure I now got to the point that in building "high 
level" packages/projects the CMake build spends a very noticeable amount of 
time in calculating the dependencies of source files. To give you a feel of the 
issue, in one example that I was profiling, I build a small "package" that uses 
>180 other "packages", which at this point are all installed on a network file 
system (AFS). Those low level packages themselves depend on N external packages 
as well (like Boost, Oracle, etc.).

So the include list for the build becomes very long, with most of the needed 
headers being outside of the source tree. In this case evaluating all the 
dependencies for a shared library made up of just 3 source files can take >10 
seconds.

I tried to find a way to tell CMake to ignore the dependencies that are outside 
of the source tree. As at this point for my use case those files can be 
considered fixed. I don't need to consider that they may change without me 
knowing. But as far as I could see, there is no mechanism in the code at the 
moment to achieve this.

I did consider teaching cmDepends to only take non-system include paths into 
account for the dependency calculation. So that whatever is specified as a 
system include path, would be ignored. But since I found that GCC becomes 
significantly slower in building my sources if I specify >100 -isystem /afs/... 
paths to it compared to specifying all those paths with -I, I gave up on this 
idea.

I also considered adding a new regular expression to cmDependsC to be able to 
specify using a regular expression which include paths to take into account and 
which to ignore. Which could still be a good idea to implement in cmDepends 
(not cmDependsC) later on, but for my own use case something simpler was enough.

In the end I applied the following patch to CMake 3.4.1 locally to speed it up 
for my use case very significantly. Of course this is not a patch that could be 
applied to CMake for a general audience. But I do think that if this 
code/behaviour could be switched on using something like a directory property / 
global variable, a lot of users could make good use of it. As it can be a 
reasonable assumption in many development environments that only the changes 
inside of the source tree should be tracked by the build system.

If you think that this is a good idea, I would be willing to provide some help 
in implementing it "properly" as well.

Cheers,
 Attila



cmDepends-sourceOnly.diff
Description: Binary data
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers

Re: [cmake-developers] Proposal to teach cmDepends to only take dependencies from the source tree

[Ben Boeckel]

On Tue, Dec 08, 2015 at 10:09:13 +0100, Attila Krasznahorkay wrote:
> In the end I applied the following patch to CMake 3.4.1 locally to
> speed it up for my use case very significantly. Of course this is not
> a patch that could be applied to CMake for a general audience. But I
> do think that if this code/behaviour could be switched on using
> something like a directory property / global variable, a lot of users
> could make good use of it. As it can be a reasonable assumption in
> many development environments that only the changes inside of the
> source tree should be tracked by the build system.

So some projects allow you to override specific headers (e.g., Boost) to
provide  which would be included from the source
tree. This file is not included directly by any users of Boost
(usually), but instead included via other Boost headers, so scanning of
system includes can be important.

So as long as there's an option/policy for it, I'm fine with the
behavior. A policy could make it the default too with a directory
property to re-enable global scanning.

Hmm…the build tree should also probably be allowed as well.

--Ben
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake-developers