I know this isn't an answer to your question, but do you have a simplified, sanitized test case yet that demonstrates the problem and could be posted publicly? - Dan
On Wed, Oct 1, 2014 at 10:44 AM, Phil Grabsky <[email protected]> wrote: > Greetings all, > We've been making an effort to optimize our builds and we seem to be > hitting the limit of what we can accomplish with makefile cleanup. I've > turned around some significant improvements by using techniques like > simplifying makefiles where possible, converting recursively-expanded > variables to simple variables, eliminating shell commands, etc, but we're > still seeing slow performance from Make itself. > > Our build is extremely parallelizable and we run with -j (unlimited) to run > as many things as possible in parallel. We use a home-grown job > distribution tool to send individual tasks to clients on other machines for > execution, so there's very little load on the machine running Make itself. > What we see is that if we have more than 30 or so clients connected to the > build Make is often unable to generate tasks fast enough to keep them busy. > > We brought in Electric Cloud to do an evaluation of their make-compatible > accelerator tool and found that it was able to instantly saturate as many > workers as we could throw at it and keep them busy for the duration of the > build, cutting the overall time nearly in half. Unfortunately, their > approach introduces a number of changes that would require significant > reworking of our build system, and more importantly they want quite a lot > of money for it (enough to hire additional senior developers for our team). > > Our management doesn't want us taking time to learn the internals of make > and see if we can improve it ourselves, but they are willing to throw money > at the problem, either by offering some sort of donation/bounty for > performance improvements or directly hiring an expert on a contract basis; > any improvements achieved would of course be submitted back to the > community as patches. > > Is there an established place to post this sort of offer? fossfactory.org > seems like what I'm looking for, but it doesn't look like it's terribly > active. > _______________________________________________ > Help-make mailing list > [email protected] > https://lists.gnu.org/mailman/listinfo/help-make _______________________________________________ Help-make mailing list [email protected] https://lists.gnu.org/mailman/listinfo/help-make
