On 07/27/2010 06:48 PM, Russell Harmon wrote: > We're looking to use cmake on a project which has some dependencies > which are not properly picked up by cmake's dependency scanner. I'm not > looking to fix the scanner, because I feel the scanner is fundamentally > broken:
Yes, it certainly has some limitations. I agree that the solution is not to fix the current scanner. I've considered using the "wave" c++ preprocessor library but it is problematic to locate all the implicit include directories and preprocessor definitions for each compiler. Using the compiler itself to get dependencies is much better. FYI, our Fortran dependency scanner does do some limited preprocessing to pick out correct Fortran 90 "use" and "module" directives. This is necessary just to generate a correct build. > In order to properly scan a file for dependencies, you need to either > use the compiler specific dependency scanner [1][2], or implement a > fully featured c preprocessor. You can't do a partial job like the > current dependency scanner does. An ancient (1990's) predecessor to CMake actually did full preprocessing of every translation unit and scanned for the "#line" directives or similar constructs in the output. The trade-off for such perfect dependencies is that it takes almost twice as long to build because every translation unit needs to be preprocessed twice. We purposely chose to use a limited scanner in CMake which is very fast and works well for many projects. Since the set of #include lines one can "grep" from a source file does not depend on the preprocessing state we can share/cache results from scanning each file. This allows us to scan all dependencies in a target a few orders of magnitude faster than the compiler takes to build it. For non-developer (end-user or packager) builds that do not need to rebuild this is very nice. The opposing trade-off is that projects that use conditional and macro-based includes do not get perfect dependencies. This has not been a problem for our needs in practice. I'm not opposed to adding support for more advanced dependency scanning in projects that are willing to pay the cost of additional scanning time for better dependencies. See below. > I'd encourage you to read "Recursive Make Considered Harmful" [3] Ironically it is impossible to do make-time implicit dependency scanning without recursive make (or include directive reload magic extensions in GNU make). After generating dependencies make needs to invoke another make recursively to load them. When we throw generated source files into the mix then we cannot even scan them until the sources have been generated. > 1. #define ARCH x86 > #define ASM_INCLUDE <ARCH/asm.h> > #include ASM_INCLUDE > cmake adds a dependency on ASM_INCLUDE instead of x86/asm.h A simple test example shows me that CMake skips this dependency rather than generating a broken build. > 2. #ifdef DEBUG > #include <debug_config.h> > #else > #include <config.h> > #endif > cmake adds a dependency on both debug_config.h and config.h > In a worst-case scenario here (one which our project has), this could > generate circular dependencies which cause the build to fail. The dependency we generate is between an object file and the headers. The dependency lines never have .h files on the left or .o files on the right, so there cannot be circular dependencies. Are you actually seeing this problem with CMake? > 3. #if 0 > #include <nonexistantfile.h> > #endif > cmake adds a dependency on nonexistantfile.h when it shouldn't have A simple test example shows me that CMake skips this dependency because it cannot find the header's location. This is true of system headers in some cases too. > My suggestion for a solution to this problem is to use the compiler > specific dependency generation methods. This is a good solution, but it should be enabled optionally for the performance reasons I describe above. > This could be done cleanly by converting dependency generation > into a plugin based system with plugins for all the supported > compilers. Yes. Howerver they should not be plugins in the dynamic loader sense. We do not want to support an SDK, require users to install extra components, or depend on the target toolchain to be capable of building a plugin that is ABI-compatible with the running CMake. This can probably be done with a simple command-line interface. We just need a way to tell CMake how to invoke a third-party tool and load dependencies from the results. I've considered doing something similar for implicit dependencies of rules defined by add_custom_command, but that is much harder because the VS IDE does not provide an easy way to implement it. This is not a problem for C preprocessor dependencies though. I envision this capability being activated by a target property which is initialized from a CMake variable, much like the RUNTIME_OUTPUT_DIRECTORY property. This makes it easy to enable project-wide or in specific targets. > I would have (tried to) made these changes to cmake already, but since > this is both a non-trivial change and an architectural change, I felt it > necessary to get an ok from you before I changed anything. I'm interested in seeing a solution to this problem. Let's discuss the design here first though. What interface do you propose? Thanks! -Brad _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the CMake FAQ at: http://www.cmake.org/Wiki/CMake_FAQ Follow this link to subscribe/unsubscribe: http://www.cmake.org/mailman/listinfo/cmake