Re: [CMake] Odd Behavior in macOS Mojave

[Steven Stallion]

On Wed, Nov 14, 2018 at 10:54 AM Sean McBride  wrote:

> I see.  Well, I just checked my macOS 10.14.1 / Xcode 10.1 bot 
> "RogueResearch12" and indeed it does not have a /usr/include folder.  It is 
> however able to build CMake, VTK, and ITK nightly without any compiler errors.
>
> Perhaps it's not so much a CMake bug as a bug in the project you're trying to 
> build?

Interesting. I'll take another look - I wonder if it's a bad
interaction from calling clang from a custom command.

Thanks for taking a look!
Steve
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Tips on reducing configure time

[Eric Noulard]

Le mer. 14 nov. 2018 à 13:25, Poughon Victor  a
écrit :

> Yes you are correct on all your observations. We already use ninja and
> ccache wherever we can. In fact we have an issue about the whole end-to-end
> build performance where we track all effort on this throughout the project
> (if you're interested:
> https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1649)
>

End-to-end build performance is a subject of interest to me, I'll have a
look.


> There are indeed quick and easy wins to be had with those tools (and we
> are working on it). But my original question is not about that. It's about
> speeding up the configure step with makefile generator. I still don't think
> it's normal that it does hundreds of thousands of I/O on files that are a
> few bytes or even empty. However it's possible that it's because we do
> something incorrect in our CMakeLists.txt and not CMake's fault.
>

I don't really know makefile-generator internals so I cannot tell.
For sur the current OTB build dir constructed with Makefile generator is
spitting out around 180 Makefile.
May be you can profile the cmake execution by building a debug version of
CMake and collect more precise insight on where the bottlenecks may reside.

You may track try_compile which could obviously be slow and may be
configure_file as well.
Otherwise I don't know, I guess you'll have to profile the cmake run.

It's true that a few minutes of configure is not much when doing a full
> build, but consider incremental builds where all you do is change one cxx
> file (and cmake is triggered because you changed git branch or something).
> Then the generate step is a significant part of the time you wait.
>

Yes that right.
Note however that changing git branch should nor per-se trigger a cmake run.
Only changing a CMakeLists.txt or configured file or explicitely specified
by
CMAKE_CONFIGURE_DEPENDS.

Otherwise changing branch with git that only modifies source shouldn't
trigger cmake rebuild.
I do that all the time in a classical git branch dev model.



-- 
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Odd Behavior in macOS Mojave

[Sean McBride]

On Wed, 14 Nov 2018 10:43:28 -0600, Steven Stallion said:

>We first noticed that some sources would fail to find certain system
>headers (stddef.h in this case) when using Xcode 10 (also seen on
>Xcode 10.1). It looks like Apple decided to relocate system include
>paths under /Library /Developer and did away with /usr/include. After
>some spelunking, it looks like this was tucked away in the release
>notes for Xcode:
>https://developer.apple.com/documentation/xcode_release_notes/
>xcode_10_release_notes

I see.  Well, I just checked my macOS 10.14.1 / Xcode 10.1 bot 
"RogueResearch12" and indeed it does not have a /usr/include folder.  It is 
however able to build CMake, VTK, and ITK nightly without any compiler errors.

Perhaps it's not so much a CMake bug as a bug in the project you're trying to 
build?

Cheers,

-- 

Sean McBride, B. Eng s...@rogue-research.com
Rogue Researchwww.rogue-research.com 
Mac Software Developer  Montréal, Québec, Canada


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] cmake 3.12 - python libs / 32 / 64 bits

[Marc CHEVRIER]

The way you proceed is wrong.
The system configuration is determined during the 'project' function call.
Setting information after this call is useless. So, in your example, on a
64bit system, the compilation configuration will be 64bit (variable
CMAKE_SIZEOF_VOID_P has value 8).
This explain why 'FIND_LIBRARY_USE_LIB32_PATHS' has no effect. This
property is taken in consideration only on 32bit systems.

Moreover, I am not sure that modifying variable 'CMAKE_SYSTEM_PROCESSOR' is
a valid action.

To configure a 32bit compilation environment, two possibilities:
* using environment variables: env CFLAGS=-m32 CXXFLAGS=-m32 cmake ...
* using a toolchain file: see
https://cmake.org/cmake/help/v3.13/manual/cmake-toolchains.7.html


Le mer. 14 nov. 2018 à 14:06, Stéphane Ancelot  a
écrit :

> Hi,
>
> My system is 64 bits but I can cross compile python c modules for 32 bits .
>
> Unfortunately I don't manage to retrieve python 32 libs , always the 64
> bits version is found.
>
>
> here is what I tried :
>
> cmake_minimum_required(VERSION 3.10)
> project(py_autom)
>
>
> set(CMAKE_SYSTEM_PROCESSOR "i686")
>
> set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -m32" CACHE STRING "c++ flags")
> set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS} -m32" CACHE STRING "c flags")
>
> set_property(GLOBAL PROPERTY FIND_LIBRARY_USE_LIB32_PATHS TRUE)
> set_property(GLOBAL PROPERTY FIND_LIBRARY_USE_LIB64_PATHS FALSE)
>
> include_directories(python2.6)
> find_package(Python2 COMPONENTS Development)
>
>
>
> message(STATUS "python ${PYTHON_INCLUDE_DIRS} ${Python2_LIBRARIES_DIR}
> ${Python2_FOUND}")
>
> Regards
>
> Steph
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> https://cmake.org/mailman/listinfo/cmake
>
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Odd Behavior in macOS Mojave

[Steven Stallion]

On Wed, Nov 14, 2018 at 8:19 AM Sean McBride  wrote:

> Could you elaborate?  What odd behaviour are you seeing?  Any error message 
> to share?  What 'workaround package' are you speaking of?  What version of 
> Xcode?

We first noticed that some sources would fail to find certain system
headers (stddef.h in this case) when using Xcode 10 (also seen on
Xcode 10.1). It looks like Apple decided to relocate system include
paths under /Library /Developer and did away with /usr/include. After
some spelunking, it looks like this was tucked away in the release
notes for Xcode:
https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes

"The Command Line Tools package installs the macOS system headers
inside the macOS SDK. Software that compiles with the installed tools
will search for headers within the macOS SDK provided by either Xcode
at:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk

or the Command Line Tools at:

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk

depending on which is selected using xcode-select.

The command line tools will search the SDK for system headers by
default. However, some software may fail to build correctly against
the SDK and require macOS headers to be installed in the base system
under /usr/include. If you are the maintainer of such software, we
encourage you to update your project to work with the SDK or file a
bug report for issues that are preventing you from doing so. As a
workaround, an extra package is provided which will install the
headers to the base system. In a future release, this package will no
longer be provided. You can find this package at:

/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg


To make sure that you're using the intended version of the command
line tools, run xcode-select -s  or xcode select -s
/Library/Developer/CommandLineTools after installing."

Installing the package seemed to resolve the issues we had building
with AppleClang, which made me wonder if something is needed in the
platform modules to account for this change.

HTH,
Steve
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Odd Behavior in macOS Mojave

[Sean McBride]

On Tue, 13 Nov 2018 18:42:43 -0600, Steven Stallion said:

>It appears there have been some changes to how macOS handles the
>system include path starting with macOS Mojave. It looks like Apple
>has provided a workaround package you can install to re-instate the
>/usr/include hierarchy, however I was wondering if this is something
>that could be handled by AppleClang support?

Could you elaborate?  What odd behaviour are you seeing?  Any error message to 
share?  What 'workaround package' are you speaking of?  What version of Xcode?

Cheers,

-- 

Sean McBride, B. Eng s...@rogue-research.com
Rogue Researchwww.rogue-research.com 
Mac Software Developer  Montréal, Québec, Canada


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

[CMake] cmake 3.12 - python libs / 32 / 64 bits

[Stéphane Ancelot]

Hi,

My system is 64 bits but I can cross compile python c modules for 32 bits .

Unfortunately I don't manage to retrieve python 32 libs , always the 64 
bits version is found.



here is what I tried :

cmake_minimum_required(VERSION 3.10)
project(py_autom)


set(CMAKE_SYSTEM_PROCESSOR "i686")

set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -m32" CACHE STRING "c++ flags")
set(CMAKE_C_FLAGS   "${CMAKE_C_FLAGS} -m32" CACHE STRING "c flags")

set_property(GLOBAL PROPERTY FIND_LIBRARY_USE_LIB32_PATHS TRUE)
set_property(GLOBAL PROPERTY FIND_LIBRARY_USE_LIB64_PATHS FALSE)

include_directories(python2.6)
find_package(Python2 COMPONENTS Development)



message(STATUS "python ${PYTHON_INCLUDE_DIRS} ${Python2_LIBRARIES_DIR} 
${Python2_FOUND}")


Regards

Steph
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Tips on reducing configure time

[Poughon Victor]

Yes you are correct on all your observations. We already use ninja and ccache 
wherever we can. In fact we have an issue about the whole end-to-end build 
performance where we track all effort on this throughout the project (if you're 
interested: https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1649)

There are indeed quick and easy wins to be had with those tools (and we are 
working on it). But my original question is not about that. It's about speeding 
up the configure step with makefile generator. I still don't think it's normal 
that it does hundreds of thousands of I/O on files that are a few bytes or even 
empty. However it's possible that it's because we do something incorrect in our 
CMakeLists.txt and not CMake's fault.

It's true that a few minutes of configure is not much when doing a full build, 
but consider incremental builds where all you do is change one cxx file (and 
cmake is triggered because you changed git branch or something). Then the 
generate step is a significant part of the time you wait.

Victor Poughon

De : Eric Noulard 
Envoyé : mercredi 14 novembre 2018 12:34
À : Poughon Victor 
Cc : CMake Mailinglist 
Objet : Re: [CMake] Tips on reducing configure time


Le mer. 14 nov. 2018 à 12:06, Poughon Victor 
mailto:victor.poug...@cnes.fr>> a écrit :
Thanks for the replies

You can see configure times on our dashboard (switch to advanced view): 
https://dash.orfeo-toolbox.org/index.php?project=OTB
It ranges from 1-3min. There's even one windows machine that's at 4m30s.

Somes windows system are known to be slow on many files accesses.
However on this particular machine 
(https://dash.orfeo-toolbox.org/buildSummary.php?buildid=342008 I guess) you 
already seem
to be using NInja + MSVC. so many small files access to ?makefiles? should not 
be the culprit.

That said whatever the system your build and test time seems to dominate the 
overall time (in the CI).

I have had great speed-up experience using CMake+ninja+ccache  for our build in 
CI. We did go from 1 hour down to less than 10min.
We only have various linux build (no Windows) it is really a big time savior 
combo.

We even share the compil' cache between the CI and the developers. So everybody 
is feeding compile cache and takes benefits from others including CI.

You seem to be using that as well here:
https://dash.orfeo-toolbox.org/buildSummary.php?buildid=341993
and you get less than 5min build time which corroborate my suggestion.

The same machine build with clang: 
https://dash.orfeo-toolbox.org/buildSummary.php?buildid=342033 is lasting 30min+
and is not using ccache.

Thanks for the tip about Ninja generator, it's indeed much faster. But 
switching the entire dev team + CI servers from make to ninja is not really an 
option for us right now because of a bug in ninja that affects our SuperBuild 
configuration (see here if you're interested in the details: 
https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1625)

I see.
Not using ninja in the Superbuild case does not mean you cannot use it in other 
cases, right?
May be the one-size fits all is a little too much to ask at this point? My 
opinion though.
Did you report the issue upstream (ninja or cmake)?

I was impressed in the past by the amount of tests orfeo is doing, I'm still 
are !!
Are you running those tests in parallel (with ctest --parallel) ? It can be a 
time savior as well is the running machine has a fair amount of cores and 
memory.


--
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Tips on reducing configure time

[Eric Noulard]

Le mer. 14 nov. 2018 à 12:06, Poughon Victor  a
écrit :

> Thanks for the replies
>
>
>
> You can see configure times on our dashboard (switch to advanced view):
> https://dash.orfeo-toolbox.org/index.php?project=OTB
>
> It ranges from 1-3min. There's even one windows machine that's at 4m30s.
>

Somes windows system are known to be slow on many files accesses.
However on this particular machine (
https://dash.orfeo-toolbox.org/buildSummary.php?buildid=342008 I guess) you
already seem
to be using NInja + MSVC. so many small files access to ?makefiles? should
not be the culprit.

That said whatever the system your build and test time seems to dominate
the overall time (in the CI).

I have had great speed-up experience using CMake+ninja+ccache  for our
build in CI. We did go from 1 hour down to less than 10min.
We only have various linux build (no Windows) it is really a big time
savior combo.

We even share the compil' cache between the CI and the developers. So
everybody is feeding compile cache and takes benefits from others including
CI.

You seem to be using that as well here:
https://dash.orfeo-toolbox.org/buildSummary.php?buildid=341993
and you get less than 5min build time which corroborate my suggestion.

The same machine build with clang:
https://dash.orfeo-toolbox.org/buildSummary.php?buildid=342033 is lasting
30min+
and is not using ccache.


> Thanks for the tip about Ninja generator, it's indeed much faster. But
> switching the entire dev team + CI servers from make to ninja is not really
> an option for us right now because of a bug in ninja that affects our
> SuperBuild configuration (see here if you're interested in the details:
> https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1625)
>

I see.
Not using ninja in the Superbuild case does not mean you cannot use it in
other cases, right?
May be the one-size fits all is a little too much to ask at this point? My
opinion though.
Did you report the issue upstream (ninja or cmake)?

I was impressed in the past by the amount of tests orfeo is doing, I'm
still are !!
Are you running those tests in parallel (with ctest --parallel) ? It can be
a time savior as well is the running machine has a fair amount of cores and
memory.


-- 
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Tips on reducing configure time

[Poughon Victor]

Thanks for the replies

You can see configure times on our dashboard (switch to advanced view): 
https://dash.orfeo-toolbox.org/index.php?project=OTB
It ranges from 1-3min. There's even one windows machine that's at 4m30s.

Thanks for the tip about Ninja generator, it's indeed much faster. But 
switching the entire dev team + CI servers from make to ninja is not really an 
option for us right now because of a bug in ninja that affects our SuperBuild 
configuration (see here if you're interested in the details: 
https://gitlab.orfeo-toolbox.org/orfeotoolbox/otb/issues/1625)


Victor Poughon

De : Eric Noulard 
Envoyé : mercredi 14 novembre 2018 11:12
À : Poughon Victor 
Cc : CMake Mailinglist 
Objet : Re: [CMake] Tips on reducing configure time

Hi Victor,
Le mer. 14 nov. 2018 à 10:20, Poughon Victor 
mailto:victor.poug...@cnes.fr>> a écrit :
Hi all

I am working on a large C++ project using CMake 
(https://www.orfeo-toolbox.org/) and our 'configure' step takes quite a lot of 
time. This can be annoying for developers working on features where the 
configure step needs to be re-run, or for CI in general.

Could you give us a clue on "takes quite a lot of time"?
Are we speaking of tens of seconds, minutes etc... ?


We have looked into it a bit and it seems like one issue is that the 
configure/generate steps do a lot of small files I/O. Specifically, using:

inotofy-wait -mr 

shows that "cmake ." causes about 150K file access/modify, etc. There are some 
especially guilty looking files like 'Makefile2.tmp' or 'progress.tmp' that 
appear thousands of times in most of the log file.

I have an SSD which somewhat helps, but for colleagues still on mechanical 
drives this is especially painful. Is there a way to disable writing all those 
files to the disk?

Any other tips on optimizing the performance of 'cmake .'?

Did you try to test drive a switch to ninja generator ?
Ninja generator generates only two files (build.ninja and rules.ninja) you 
won't have the "mny small files issue".
You can have long configuration times as well but not coming from file access.




Thanks,

Victor Poughon


--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake


--
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Tips on reducing configure time

[Cristian Adam]

On Wed, Nov 14, 2018 at 10:20 AM Poughon Victor 
wrote:

> Hi all
>
> I am working on a large C++ project using CMake (
> https://www.orfeo-toolbox.org/) and our 'configure' step takes quite a
> lot of time. This can be annoying for developers working on features where
> the configure step needs to be re-run, or for CI in general.
>
> We have looked into it a bit and it seems like one issue is that the
> configure/generate steps do a lot of small files I/O. Specifically, using:
>
> inotofy-wait -mr 
>
> shows that "cmake ." causes about 150K file access/modify, etc. There are
> some especially guilty looking files like 'Makefile2.tmp' or 'progress.tmp'
> that appear thousands of times in most of the log file.
>
> I have an SSD which somewhat helps, but for colleagues still on mechanical
> drives this is especially painful. Is there a way to disable writing all
> those files to the disk?
>
> Any other tips on optimizing the performance of 'cmake .'?
>
> Thanks,
>
> Victor Poughon
>
>
Hi,

if a subsequent CMake  run is faster than the initial one, you can have a
look at
https://github.com/cristianadam/cmake-checks-cache

If you cache the results of the configure checks you can have the initial
run as fast as subsequent ones!

Cheers,
Cristian.
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

Re: [CMake] Tips on reducing configure time

[Eric Noulard]

Hi Victor,

Le mer. 14 nov. 2018 à 10:20, Poughon Victor  a
écrit :

> Hi all
>
> I am working on a large C++ project using CMake (
> https://www.orfeo-toolbox.org/) and our 'configure' step takes quite a
> lot of time. This can be annoying for developers working on features where
> the configure step needs to be re-run, or for CI in general.
>

Could you give us a clue on "takes quite a lot of time"?
Are we speaking of tens of seconds, minutes etc... ?


>
> We have looked into it a bit and it seems like one issue is that the
> configure/generate steps do a lot of small files I/O. Specifically, using:
>
> inotofy-wait -mr 
>
> shows that "cmake ." causes about 150K file access/modify, etc. There are
> some especially guilty looking files like 'Makefile2.tmp' or 'progress.tmp'
> that appear thousands of times in most of the log file.
>
> I have an SSD which somewhat helps, but for colleagues still on mechanical
> drives this is especially painful. Is there a way to disable writing all
> those files to the disk?
>
> Any other tips on optimizing the performance of 'cmake .'?
>

Did you try to test drive a switch to ninja generator ?
Ninja generator generates only two files (build.ninja and rules.ninja) you
won't have the "mny small files issue".
You can have long configuration times as well but not coming from file
access.




>
> Thanks,
>
> Victor Poughon
>
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> https://cmake.org/mailman/listinfo/cmake
>


-- 
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake

[CMake] Tips on reducing configure time

[Poughon Victor]

Hi all

I am working on a large C++ project using CMake 
(https://www.orfeo-toolbox.org/) and our 'configure' step takes quite a lot of 
time. This can be annoying for developers working on features where the 
configure step needs to be re-run, or for CI in general. 

We have looked into it a bit and it seems like one issue is that the 
configure/generate steps do a lot of small files I/O. Specifically, using:

inotofy-wait -mr 

shows that "cmake ." causes about 150K file access/modify, etc. There are some 
especially guilty looking files like 'Makefile2.tmp' or 'progress.tmp' that 
appear thousands of times in most of the log file.

I have an SSD which somewhat helps, but for colleagues still on mechanical 
drives this is especially painful. Is there a way to disable writing all those 
files to the disk?

Any other tips on optimizing the performance of 'cmake .'?

Thanks,

Victor Poughon


-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
https://cmake.org/mailman/listinfo/cmake