Hi,
On Fri, Oct 16, 2009 at 07:29:30PM +0200, Craig wrote:
> Hi,
>
> a patch (set weight/get weight) I imagined some days ago was integrated
> just 6hrs after I had thought about it (Willy must be reading them!).
hehe :-)
> I've written a simple (exchangable) interface that prints out a servers
> load and a client to read it. I plan to read the load from all servers
> and adjust the weight dynamically according to the load so that a very
> busy server gets less queries and the whole farm is more balanced. I
> plan to "smoothen" the increasing/decreasing a bit so that there aren't
> too great jumps with the weight, I want to implement a policy of
> something like "oh that server can do 50% more, lets just increase the
> weight by 25% and check again in a minute". I hope this will autobalance
> servers with different hardware quite well, so that you don't have to
> guess or do performance tests to get the weights properly.
>
> Some python code is already finished (partly because I'd like to
> practise a bit) but I didn't continue yet, because I'd like to hear your
> opionions about this.
>
> Am I mad? ;)
No you're not mad, this is a subject that some people regularly bring
back on the table. However you should consider a few things :
- report server idle instead of load. It's better because it's
always bound by 100%, and you can later modulate it by multiplying
or bounding it by other criteria (eg: %free RAM).
- make it optionnally modulate the maxconn, just like the slowstart
does. This is important too, because if your server is saturated,
better not add more requests to it and queue them.
- set the measured weight as another factor to compute the effective
weight, and do not replace any user weight. That way you can still
let the admin tweak the server's weights if they want more load on
some servers than others.
- make the check header name for response configurable. That way,
when we later report captured some headers on the stats page, it
will be possible to display the server's idle too.
- be very careful when playing with weights, as there is always a
risk of drifting too much towards any direction. Ideally, you
should always base your computations on the next 2 measures and
place the cursor accordingly. For instance, if you had 25% then
increase the weight and get 80%, and you can find a rule to
quickly and safely converge towards 50% usage.
- provide the ability to set an idle goal in the configuration.
You don't want less, you don't want more.
- be careful about CPU usage jumps, eg during log compression or
things like this which can eat all of the CPUs multiple times
and make the algorithm take some time to recover.
BTW, this may be a bit off-topic, but still related to the subject.
I was thinking about modulating weights based on the server queue
sizes. The idea would be to find an algorithm that tries to reduce
the server's queue weight to the smallest possible value, meaning
it would improve response times. This is hard though because it
would only take into account visitors tha are already assigned to
a server, and might be a bit jerky. But this was just a thought I
wanted to share.
Regards,
Willy
