Hi,
We use distcc here at paypal, with many developers compiling,
i.e. preprocessing and linking, on 4 machines with a shared disk. The
file locking seems work pretty well, even on a shared filesystem.
We have made some changes to distcc to cope with our environment, and to get
better performance. I’d like to start submitting the more
general changes back to the source tree.
- A change to make preprocessing
grab a lock file before commencing with the job. This allows
much more parallel make with minimal performance impact, because the extra
jobs just spin waiting for a lock. Without this change, the extra
jobs all compete for cpu time, and slow each other down, all while holding
backend locks. This greatly streamlines the process, and
reduces the number of locks held.
- Randomizing the host selection
algorithm. This randomized based on slots instead of
hosts. The advantage here is that a host with more slots is more likely
to be chosen. Our backend farm has about ten 4 processor
machines, and about 30 single processor windows boxes. The
existing algorithm always fills up the windows machines before using extra
capacity in the multi-cpu boxes. By changing how the hosts are
selected, the multi-processor boxes are now 4 times more likely to be
selected, which matches their 4x of capacity.
I would like to get these changes cleaned up, and
parameterized so that they can be turned off and controlled, but once that is
done, what is the preferred way of submitting patches?
Michael
|
__
distcc mailing list http://distcc.samba.org/
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/distcc
