As brought up earlier, I'm unhappy with the current automake build system used in VOS due to maintainance overhead (even trivial modules have grown to require a dozen lines of boilerplate) and the considerable difficulty it poses to properly supporting Windows development. We resolved the last discussion with "needs more research". Well, I've done some research.
I've decided on the following criteria to look for in a build system: - Free software (obviously) - Cross platform (must support Linux/OS X/Windows, gcc and MSVC) - Build files arn't too verbose - Build files can be located in the same directory as the source code for each subproject (library or app) - Can add new rules that apply across the whole project without having to edit every build file individually - "configure" ability, or easy to integrate with autoconf - Supports robust parallel builds - Out-of-tree builds - Something along the lines of "make install", "make dist" and "make distcheck" Here's what I'm currently looking at, grouped into two caterogies: Makefile generators: * Bakefile We talked about this one before. This tool was developed for WxWidgets and supports a long list of compilers and backend build systems. It describes the build using a XML file. A build rule inherits from a "template" to supply environment parameters (such as debug or not) and templates can extend other templates, which seems fairly powerful. By having a few common "root templates", it it easy to apply changes across the entire project. The user can also declare certain kinds of conditional expressions based on parameter values. Bakefile creates Makefile.in files and integrates nicely with autoconf, which means the user does not need to have bakefile installed if they only need to compile. Because it uses other tools for the actual build, parallel build support is only as good as the underlying tool. I'm not sure how bakefile handles subprojects. I suspect you need to set up a toplevel build file and use the <include> feature to bring in each separate project file, which results in a monolithic Makefile (this does suggest parallel builds are more likely to work). There does not appear to be any "make dist" support, but there is "make install" support. * CMake Another project file generator. This one was developed for VTK (the Visualization Toolkit) and is also notable for being the build system used by KDE. It uses its own simplistic syntax for describing build rules. It has parameter substitution and conditionals. It has commands like FIND_LIBRARY() that take care of some of the responsibilities of autoconf, but I believe these checks are done at the point that you run "cmake", so you can't distribute the resulting Makefile if they have system-specific configuration. This means the user needs to have cmake installed. I'm not sure how you would extend a build rule across an entire project, possibly using ADD_CUSTOM_COMMAND() to set up a customized rule. Subprojects are easy, each file specifies what subdirectories to search for further build rules. Makefiles are output in each directory separately, and it uses recursive make (which mean parallel builds are out). There is no "make dist" support, but there seems to be "make install" support. Make replacements: * Perforce Jam Jam is a make replacement used by several projects, most notably for us Crystal Space (as such I've had more direct experience with it than the others). It provides a way of specifying a rule and a target (generally an output library or application) and specifying attributes that goes with that target (such as other libraries to link to). It has parameter substitution and conditionals. Customization seems to be done mainly through customizing build rules. A build rule is defined using Jam's own scripting language which seems to be an interesting mix of declarative referencing of other rules and imperative statements (conditionals and variable assignments). Jam uses a monolithic build, but allows the user to specify individual jamfiles recursively (each jamfile specifies a list of subdirectories to look in for more jamfiles). Somewhat obnoxiously, each jamfile must explicitly specify the path from the root directory to the directory the jamfile is in (I don't understand why it can't figure this out for itself.) Because it computes a gloal dependency tree, it does a good job with parallel builds. Jam seems to be more reliant on user customization than the others, with a very minimal default rule set. However, if we were to use Jam, we would probably want to adopt an existing build system with a robust set of rules, such as the Crystal Space build system. Jam can used with autoconf, by using a files like "Jamconfig.in" that lets autoconf fill in all its relevant findings. Because Jam rules abstract out compiler-specific settings, it is possible to use Jam (with sufficient application of custom rules) as a makefile generator as well. Crystal Space produces both makefiles and Visual Studio projects from its jam-based build system. * Boost.Build (bjam) Boost Jam is a fork of Perforce Jam adopted by the Boost project. There are two flavors, known as Boost.Build v1 and v2. Boost jam is designed to be used specifically with the Boost ruleset, so it is most comparible to the combination of Perforce Jam + Crystal Space rules. The Boost build system provides support for many different compilers, supports out-of-tree builds and has rules for both "staging" (copying binaries to an intermediate directory, similar to what the VOS "inplace/" directory tree) and installing. Although derived from Jam, bjam seems to have a different syntax for target attributes. Notably different from Perforce Jam (so far as I can tell) it supports rule templates so the user can specify a set of attributes to be used across different targets, and templates can derive from other templates. Boost itself uses autoconf for features tests, but you wouldn't know it because the boost distribution by default is preconfigured for a variety of platforms so it is rarely required to do the full autoconf check. So, the discussion about autoconf and jam above presumably holds for Boost Jam as well. The discussion above about the monolithic build and good support for parallel builds also holds. Unlike Crystal Space, which supports MSVC through generating Visual Studio project files, Boost supports MSVC by requiring the user go to the cmd shell and running bjam.exe which runs the compiler itself. This has the advantage of ensuring everyone is always using the same build system, but it can be very annoying to Windows users used to using the IDE for everything. * Scons Scons is, supposedly, the "next big thing". Scons is, at its heart, a robust dependency engine written in Python. Scons files are actually Python scripts that call functions declaring targets and dependencies; these scripts can access the entire Scons API as well as perform any Python action. This grants the developer an unusual level of flexibility in scripting builds, since arbitrarily complex logic is available. The Scons developers focused strongly on providing a robust solution to the most basic task of the build system, which is to figure out what needs to be built, and to build it, and never do any extra work or (worse) fail to rebuild when something has changed. Unlike most other build systems, dependencies and rebuild requirements in Scons are based on file hashes rather than timestamps, which gives much stronger guarantees about whether a file has changed on not and needs to be rebuilt. Scons is said to have terrific parallel build support, as well as distcc support (enabling multi-host parallel compiles). Scons provides feature test functions to fill the role of autoconf. It supports hierarchical builds, allowing the toplevel file to either list all subprojects, or recursively. In either case, the result is still a monolithic build where all dependencies are scanned and computed before compiling begins. Scons has install support, but no dist support. I'm not sure if it supports out-of-tree builds (probably, but not obvious from the documentation.) Scons has a notion of "construction environments" which provide parameters (such as CFLAGS) to build rules. This appears to be similar to the templating features described earlier for Bakefile and Bjam. "Deriving" an environment seems to be a copy-and-modify operation done at the procedural Python level. One feature unique to Scons is support for checking out code from source control systems. This is also interesting since both Scons and Bzr are written in Python, we might be able to integrate them in interesting ways. Scons is probably the most interesting of these packages in terms of approach and capabilities. Unfortunately, I also found a number of comments by people in charge of large projects that had tried Scons and found some problems: - Scons is slow. Very, very slow. It has to parse and evaluate all its Python scripts every time, and then compute hashes for every file before it can do anything. It also runs the "configure" feature tests every time. - Failure modes are apparently quite cryptic, and indicated with a Python exception and a stack trace rather than a nice error message. - Being that input files are actually Python scripts, they require some extra quoting which can be non-obvious for people who don't know Python. * Good old make Finally, to round out the discussion, it bears mentioning that some projects (Mozilla, for example) use autoconf and handwritten makefiles (well technically Makefile.in). These makefiles are written to deal with platform issues (microsoft compiler vs. gcc and such) but such solutions essentially are reinvented for each project. This also requires people do command line builds, even on Windows (? unless some way of generating visual studio projects is also written). Files written this way tend to require a lot of boilerplate code, as well as possibly having to maintain the dependency graph by hand. This is the default position that all the other build tools and makefile generators are trying to improve upon. All of the packages reviewed here definitely have both advantages and disadvantages, and as such no tool clearly wins out. I have not decided which way to go yet, but could see us using any of these packages. Thoughts? -- [ Peter Amstutz ][ [EMAIL PROTECTED] ][ [EMAIL PROTECTED] ] [Lead Programmer][Interreality Project][Virtual Reality for the Internet] [ VOS: Next Generation Internet Communication][ http://interreality.org ] [ http://interreality.org/~tetron ][ pgpkey: pgpkeys.mit.edu 18C21DF7 ]
signature.asc
Description: Digital signature
_______________________________________________ vos-d mailing list [email protected] http://www.interreality.org/cgi-bin/mailman/listinfo/vos-d
