Hi Bill:
Thanks for your response. More comments below.
On 2011-06-30 22:03-0400 Bill Hoffman wrote:
On 6/30/2011 5:23 PM, Alan W. Irwin wrote:
To help with the general difficulty of getting dependencies right,
would it be possible to implement a CMake change so that for a given
flag (say --check-dependencies) the cmake application issued warnings
for file and target dependency issues that would cause problems for
parallel builds for a given CMake-based build system? That would help
a lot to solve this topic that keeps coming up again and again on this
list and also make existing CMake-based build systems much easier to
maintain.
Assuming such a change was implemented, then the standard FAQ response
to this topic would then boil down to "run cmake with the
--check-dependencies flag to find out the dependency issues you have
to solve before you can reliably run a parallel build."
If we had a magic flag that did --check-dependencies we would just auto add
the depends.
I agree that a magic flag to automatically generate all depends is a
difficult/impossible thing to implement. But to be fair that is a
much tougher problem than checking an existing chain of dependencies
for validity.
The problem is these depends can be hard to find. Some of them
can not even be found until everything has been built or at least all files
have been generated, and all source files have been scanned for depend
information. CMake saves the depend generation of source files to build
time. So, at CMake time we don't even know which source files include which
other source files. This stuff can be nested as well. You could generated
foo.c which includes bar.h which is also generated right after car.h which is
included by car.c all of which might be in different targets and directories.
So, although all things are possible, this one is very hard to do....
I have had an idea based on your description of why it is essentially
impossible to check the validity of dependencies automatically at
cmake time. Could this check be implemented instead at run time when
all the dependencies are known?
One relatively simple possibility is an optional run-time check that a
given custom command is never duplicated by any particular parallel
build. (Although not a comprehensive check for bad dependencies, this
idea would already give much more reliable detection of dependency
issues than relying on build errors to find the problem for you since
re-running a custom command might or might not generate a build error
depending on whether the duplicate commands are run sufficiently close
in time by the parallel build.) Other more comprehensive run-time
checks for dependency issues might be possible as well.
If it turns out that a run-time dependency check of any kind could be
implemented, that would shore up what I consider to be the principal
weakness of CMake-based build systems which is they can have subtle
dependency clashes which only give intermittent and hard-to-reproduce
parallel build problems. For example, I have discovered a number of
dependency clashes for the PLplot build system in the past by varying
the build timing with different GNU make -j parameters until I
discovered a build error. But such checking can take a lot of time
and can never be exhaustive even on one platform and may miss build
issues that would occur on other platforms because of different
timing. That is why I would welcome any way to improve dependency
checking at run time.
Alan
__________________________
Alan W. Irwin
Astronomical research affiliation with Department of Physics and Astronomy,
University of Victoria (astrowww.phys.uvic.ca).
Programming affiliations with the FreeEOS equation-of-state implementation
for stellar interiors (freeeos.sf.net); PLplot scientific plotting software
package (plplot.org); the libLASi project (unifont.org/lasi); the Loads of
Linux Links project (loll.sf.net); and the Linux Brochure Project
(lbproject.sf.net).
__________________________
Linux-powered Science
__________________________
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the CMake FAQ at:
http://www.cmake.org/Wiki/CMake_FAQ
Follow this link to subscribe/unsubscribe:
http://www.cmake.org/mailman/listinfo/cmake
