|
As mentioned last week, I’ve put together these
changes for submission to the project. Our local setup is that no compiling is done locally.
We’ve got big enough jobs that the local machine is busy with linking and
preprocessing. We also found that going above about –j 15 didn’t
improve total time to compile by much, even on a 4 processor box. Limiting the preprocessor helps achieve a healthy level of
parallelism. Currently, we have to carefully adjust the parallel
factor in make, so that it doesn’t overwhelm the local machine. By
using lock files to control that instead, we can now dramatically increase the
j factor, and it doesn’t slow down the build. The other change is to randomize host selection, and choose
based on slots. This means that hosts with 4 available slots are 4 times
more likely to be chosen than a host with only 1 available slot. Last, the localhost lock file now includes part of the host
name. We have 4 “localhosts” in use here on shared disks. Without
this change, they all used the same lockfiles. This meant that The randomized host selection is turned on by adding the
host “—randomize” to the list. Local compilation slots can be set without adding localhost
to the hostlist, by setting “—localslots=<num>” And the preprocessor limit can be set by “—localslots_cpp=<num>”
It currently defaults to 8. I haven’t
done any performance tuning, but 8 does significantly better than “unlimited”
on our 4 processor boxes. This patch is against the 2.18.3 tarball. Our original
changes were against 2.11.2, and I decided to just reimplement them against a
clean 2.18.3 tree. We have other changes, but I’m not sure how general
they are. One outstanding problem is that with multiple “localhosts”
using 1 shared directory, orphan detection doesn’t work so well. We’ve
just turned it off, but that results in many orphans. Michael |
distcc-changes-2005-03-28
Description: distcc-changes-2005-03-28
__ distcc mailing list http://distcc.samba.org/ To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/distcc
