* Xan Lopez wrote on Fri, Jan 07, 2011 at 12:18:33AM CET: > On Thu, Jan 6, 2011 at 11:08 PM, Ralf Wildenhues <[email protected]> > wrote: > >> If I'm right this means I've moved from having the vast majority of > >> the time in user CPU time to a 60/40 split, which I guess means by now > >> we are spending a lot of time stating files (?). > > > > Can you send sysperf output for this as well, please? Thanks. > > Here it is http://people.gnome.org/~xan/make2.txt.gz
Thanks. > > The makefile + included files that I've looked at (where not all of > > webkit is built) already weigh in at some 90 MB total size, the very > > bulk of which in the .deps/*.P*o files just as you mentioned. Getting > > that down to 50some MB by not using -MP seems desirable iff make really > > is spending most of its time reading, parsing, and not stating. > > Of course it also seems desirable to not lose the deleted header hack. > > Perhaps a silly question: could it be easier to just post-process the > include of all the Plo files to get rid of the duplicates? Since the > vast majority of the data is duplicated among the files it seems a > reasonable thing to do. Sure, this seems like a viable workaround when you don't hack on Automake. However, even just generating 45MB of data only to remove it again with sed or awk takes time during building, and I'd like to avoid that time too if possible. Further, for some non-GCC compilers, the depcomp script generates all the dummy dependencies manually, avoiding that will save at least one sed per compiler invocation, and that's an advantage as well, even if the relative savings are small compared to the time a full build takes. > > For the remaining 14s, there is one more uuugly hack I can think of: > > get rid of the recursive make invocation that is done in the 'all' rule. > > # Ugly hack: override automake's definition of 'all' in order to > > # avoid a make recursion. > > # This requires order-only prerequisites (GNU make 3.80 or newer). > > all: all-am > > %.$(OBJEXT): | $(BUILT_SOURCES) autotoolsconfig.h > > %.lo: | $(BUILT_SOURCES) autotoolsconfig.h > > I've tried this and I see no significant win, perhaps a solid 0.2, > 0.15 difference (the timing is variable so it's difficult to measure > with precision wins that small). Ah, but that's only because of another bug I didn't look for: automake outputs its own 'all' rule unconditionally, i.e., even if you choose to override it. That may have been done in order to "help" those poor users adding prerequisites to 'all' instead of using 'all-local', but it hurts for this hack. Can you, just for the sake of measuring performance, edit the generated GNUmakefile and remove Automake's 'all' rule from it; something like sed -i '/^all: .*BUILT_SOURCES/,/^$/d' GNUmakefile It should really prove a big, not a small win, otherwise it's not worth looking at this further. > > BTW, it seems webkit also has a cmake-based build system (right?). > > Does what it creates fare better? > > I haven't tested it personally, but I can ask. What I know is that > Chromium uses gyp, which on Linux generates Makefiles, and they claim > their null-build time is pretty much zero (not sure on which machine, > though, so perhaps that's only the case for some huge iron). So it > would seem that whatever our problems GNU make shouldn't be one of > them. > > What I can tell you is that our autotools setup is known for being the > slowest of the lot :/ (vs Xcode, VisualStudio, CMake, Gyp, ...) Sure, we know that. ;-) We provide more portability. A null time of zero, even if only on big iron, would indicate a beta build system though, or at least some parallelization at the stat level already (if no notify daemon is used). Thanks, Ralf
