Some months ago I mentioned the world of hurt which comes out of the uts
makefiles using lots of wildcards to find source files.
Last night I got inspired, did some prototyping, and I now have real
data on how much benefit we get from making the wildcards go away.
While what I have is not even close to complete, the prototype has
already successfully built all sparc kernel modules (admittedly, I was
holding it together with the moral equivalent of duct tape and
clothespins. It flew to pieces when I attempted a nightly, but I
expected that). But the performance improvement is so dramatic I really
had to share this now..
One key microbenchmark relevant to developer productivity is the time it
takes to do an incremental build that doesn't actually build anything.
Currently, on a 8x2x1.35ghz V890, a null build of uts/sun4u/genunix
takes:
dmake 30.06s user 2.54s system 88% cpu 36.815 total
With my prototype (which constructs a genunix which links
successfully...) I have it down to:
dmake 2.65s user 0.41s system 81% cpu 3.735 total
More importantly, a complete no-op build in usr/src/uts on the
aforementioned V890 takes just under 3 minutes. And that's without
Sasha's changes to increase parallelism.
What's going on? Well, the hudnreds of wildcard rules in the various
Makefile.rules cause dmake to go hunting through hundreds of source
directories for each source file.
If you expand the wildcards once -- which is straightforward to get 95%
right with a perl script because the *.rules files are so stylized --
you eliminate all that hunting around.
As always, there are a few weirdo special cases that need manual
cleanup; it also turns out that a bunch of the wildcard rules are either
duplicates or completely pointless (no source files matched).
I filed
6613884 Wildcards in uts/.../Makefile.rules result in O(n^2) build
behavior
Oh, in case anyone's wondering, it looks like my changes don't touch any
of the makefiles Sasha is changing.
Making this real:
I'm going to take the "mkrules" script, and split it in two.
The first half will take a Makefile.rules file and generate an condensed
intermediate Makefile.rule.in file which will be human-maintainable and
which can be fed to the second script.
The second half will take the *.in file and expand all the wildcards
into a "Makefile.rule" file.
This project would:
add the scripts
add the Makefile.rule.in files
add rules to the top-level uts/Makefile to generate
Makefile.rule files
change everything that includes Makefile.rules to include
Makefile.rule instead
delete the Makefile.rules files.
The reason for the rename to Makefile.rule is to avoid causing trouble
for projects which have unintegrated changes to Makefile.rules files
when this project integrates.
- Bill
_______________________________________________
tools-discuss mailing list
[email protected]
