Greetings all, We've been making an effort to optimize our builds and we seem to be hitting the limit of what we can accomplish with makefile cleanup. I've turned around some significant improvements by using techniques like simplifying makefiles where possible, converting recursively-expanded variables to simple variables, eliminating shell commands, etc, but we're still seeing slow performance from Make itself.
Our build is extremely parallelizable and we run with -j (unlimited) to run as many things as possible in parallel. We use a home-grown job distribution tool to send individual tasks to clients on other machines for execution, so there's very little load on the machine running Make itself. What we see is that if we have more than 30 or so clients connected to the build Make is often unable to generate tasks fast enough to keep them busy. We brought in Electric Cloud to do an evaluation of their make-compatible accelerator tool and found that it was able to instantly saturate as many workers as we could throw at it and keep them busy for the duration of the build, cutting the overall time nearly in half. Unfortunately, their approach introduces a number of changes that would require significant reworking of our build system, and more importantly they want quite a lot of money for it (enough to hire additional senior developers for our team). Our management doesn't want us taking time to learn the internals of make and see if we can improve it ourselves, but they are willing to throw money at the problem, either by offering some sort of donation/bounty for performance improvements or directly hiring an expert on a contract basis; any improvements achieved would of course be submitted back to the community as patches. Is there an established place to post this sort of offer? fossfactory.org seems like what I'm looking for, but it doesn't look like it's terribly active. _______________________________________________ Help-make mailing list [email protected] https://lists.gnu.org/mailman/listinfo/help-make
