>From your description what seems to be ailing your build systemis the >architecture of your makefile itself. Without any clues as to what ails your build (since you didnt provide the makefile), you need to make your makefile non-recursive in order to reap the speed dividends. The changes that you talk about (simply-expanded variables, no shell commands, etc.) go only so far in gaining speed. For more help you can look up the Miller paper on non-recursive makefiles which explains the reasons for slow/inaccurate builds & also how to overcome them. HTH -Rakesh
> Date: Wed, 1 Oct 2014 13:44:02 -0400 > Subject: Looking for help improving performance (advice or potential contract > position) > From: [email protected] > To: [email protected] > > Greetings all, > We've been making an effort to optimize our builds and we seem to be > hitting the limit of what we can accomplish with makefile cleanup. I've > turned around some significant improvements by using techniques like > simplifying makefiles where possible, converting recursively-expanded > variables to simple variables, eliminating shell commands, etc, but we're > still seeing slow performance from Make itself. > > Our build is extremely parallelizable and we run with -j (unlimited) to run > as many things as possible in parallel. We use a home-grown job > distribution tool to send individual tasks to clients on other machines for > execution, so there's very little load on the machine running Make itself. > What we see is that if we have more than 30 or so clients connected to the > build Make is often unable to generate tasks fast enough to keep them busy. > > We brought in Electric Cloud to do an evaluation of their make-compatible > accelerator tool and found that it was able to instantly saturate as many > workers as we could throw at it and keep them busy for the duration of the > build, cutting the overall time nearly in half. Unfortunately, their > approach introduces a number of changes that would require significant > reworking of our build system, and more importantly they want quite a lot > of money for it (enough to hire additional senior developers for our team). > > Our management doesn't want us taking time to learn the internals of make > and see if we can improve it ourselves, but they are willing to throw money > at the problem, either by offering some sort of donation/bounty for > performance improvements or directly hiring an expert on a contract basis; > any improvements achieved would of course be submitted back to the > community as patches. > > Is there an established place to post this sort of offer? fossfactory.org > seems like what I'm looking for, but it doesn't look like it's terribly > active. > _______________________________________________ > Help-make mailing list > [email protected] > https://lists.gnu.org/mailman/listinfo/help-make _______________________________________________ Help-make mailing list [email protected] https://lists.gnu.org/mailman/listinfo/help-make
