Martin Pool wrote: >I'm not saying it's impossible, but it seems quite hard. If you >invent a better load spreading algorithm of course I would be happy to >hear of it. Having a central controller is a more complex design, so >I don't want to use it unless it will clearly perform better. > > Mmm, we've chosen to use LVS for this, as I've previously said, and I think this is probably a good generic way to solve this sort of problem - the loadbalancing overhead is extemely low - the most efficient mode of operation just changes the MAC address of inbound packets so that TCP connection x packets are steered to machine y (the replies go directly from distcc server to distcc client, missing out the central server), all work is done in kernel mode, so it should scale very well for use a large sites.
We have a couple of perl scripts, which we use in conjunction with mon, and LVS to move boxes in/out of the build cluster, and we are thinking of extending this to modify the individual machine weightings (i.e. new job priority), depending on such factors as load, cpu usage etc.. I would think we'll get into all sorts of dynamic load balancing issues such as setting up oscilations in load, plus the problem adding additional load to the system as a whole. Perhaps perl or python would be good choices for this component, as the work isn't time critical, and both have snmp client libraries (prob best way of checking the status of machines in the cluster) - also it should be possible to do all of this work independently of distcc (thus both keeping distcc simple for the simple usage case, and making the system generic so it can by applied to other solutions which use LVS, such as web serving). Tim. _______________________________________________ distcc mailing list [EMAIL PROTECTED] http://lists.samba.org/cgi-bin/mailman/listinfo/distcc
