Alan Burlison wrote: > Ian Collins wrote: > >> As I've suggested before, the best way I can see to improve build times >> is to "flatten out" the build. One way to do this is to create higher >> level makefiles with targets from several directories. This reduces the >> number of bottlenecks. >> >> This is how I structure my own code and I can maintain a steady number >> of jobs over several build machines this way. >> >> This will also help to bring distributed building to ON. > > > As Stephen Hahn has already pointed out I did a significant amount of > work in this area in the past. I too incorrectly assumed that the > thing slowing down builds was lack of parallelism in the makefiles, > and I was wrong as well. The actual cause was unnecessary /bin/sh > invocations which were chewing up vast amounts of CPU, and I suspect > that there is still work to be done in that area. This was directly > contrary to my expectations when I started the work, I thought the > problems would be primarily related to poor parallelism in the build. > > Please read the paper. > > http://mediacast.sun.com/share/sch/d-exacct-builds-suntech-2001.pdf > > Moral of the story: Instrument and measure before suggesting > solutions, otherwise you are probably fixing the wrong thing. > An interesting paper.
The conclusion I draw from it is that the biggest problem is the uneven distribution of jobs throughout the build process. I would be interested in seeing data for the number of instances of cc over time. I have found it best to consider parallel/distributed builds from day one and to continuously monitor builds for any bottlenecks. My process is crude, simply track the number of jobs reported by dmake and the CPU load of the build machines. I also periodically experiment with the jobs/CPU ratio for my build servers as the nature of the source files changes. I have also found that splitting (or occasionally combining) source files can have a significant impact. Ian. _______________________________________________ tools-discuss mailing list [email protected]
