RE: dynamic weights based on actual server load

[Angelo Höngens]

Just read this thread, and I thought I would give my humble opinion on this:

As a hosting provider we use both windows and unix backends, en we use haproxy 
to balance requests across sites on a per-site backend (with squid in front of 
haproxy). What I would love to see, is dynamic balancing based on the 
round-trip time of the health check.

So when a backend is slower to respond, the weight should go down (slowly), so 
the faster servers would get more requests. Now that's a feature I'd love to 
see.. And then there would not be anything to configure on the backend (we 
don't always have control over the backend application)

Just my 2 cents.
-- 

 
With kind regards,
 
 
Angelo Höngens
 
Systems Administrator
 
--
NetMatch
tourism internet software solutions
 
Ringbaan Oost 2b
5013 CA Tilburg
T: +31 (0)13 5811088
F: +31 (0)13 5821239
 
mailto:a.hong...@netmatch.nl
http://www.netmatch.nl
--

-Original Message-
From: Willy Tarreau [mailto:w...@1wt.eu] 
Sent: vrijdag 16 oktober 2009 22:59
To: Hank A. Paulson
Cc: Craig; haproxy@formilux.org
Subject: Re: dynamic weights based on actual server load

On Fri, Oct 16, 2009 at 01:36:52PM -0700, Hank A. Paulson wrote:
 For the code you are developing, if you make the interface general enough 
 so that parameters can be added or removed that would be good. 
 Telnet/text/memcached style protocols seem popular to allow easy 
 debugging/monitoring.
 So if your protocol says a machine has to send a load info bundle like:
 SS:8cbed340118ddf87e2d8ca4352006572
 SYSID: blah1
 SAMPLETIME: 2009-10-14-22-00-03
 CPU: 83.23343455
 NETI: 134238.0232
 NETO: 492283.6549
 DISK: 433.232
 ES:8cbed340118ddf87e2d8ca4352006572

It's a lot better to return these information in HTTP headers, because
that can be added on top of any other resource that haproxy would check.

(...)
 It is probably possible to just modify feedbackd to emit haproxy set 
 weight commands.

Don't do that ! The set weight is for the human, not for an automaton.
The most interesting usage is set weight xx/xx 0 to disable a server
before operating on it, or set weight xx/xx 50% to off-load it a bit
if you see it start swapping. If you send an automated tool on it, it
will constantly change your manually assigned values.

 More interesting, I think would be to combine a multiple load parameter 
 (active connections, CPU, net in/out bytes, net in/out packets, disk io, 
 etc) feedback system with the ideas from the NetBSD neural network 
 scheduler, creating an ai based dynamic load balancing system.
 http://softlayer.dl.sourceforge.net/project/nnsched/docs/thesis/nnsched.pdf
 This is more possible now that we have multi core systems that would have 
 some idle CPU resources available for the ai compute load.

if doing some computations on that few parameters consumes even
a measurable amount of CPU, then they are clearly wrong. We're
not doing fractals here. Combining 3-4 parameters should not
results on billions of operations.

Willy

Re: Redirect_prefix question

[Chris Sarginson]

Unfortunately there are quite a few subdomains, and we don't know them  
all as this is for a client, who will just be looking to add dns  
records and vhosts on the fly. What sort of work is required to add in  
this kind of functionality?

I've seen your previous warnings about 1.3.19 :)

Cheers
Chris

Sent from my iPhone

On 16 Oct 2009, at 22:09, Willy Tarreau w...@1wt.eu wrote:


Hi Chris,

On Fri, Oct 16, 2009 at 02:50:44PM +0100, ch...@sargy.co.uk wrote:


Hi Guys,

I have a wildcard SSL certificate which I wish to do a redirect from
http to https whilst maintaining the original subdomain requested,  
eg:


http://sub.domain.com/page should be redirected to
https://sub.domain.com/page

I can't do this with redirect_prefix (it seems to require I know the
destination in advance)

Can anyone point me in the right direction for this?


This would mean extract part of the Host: request header to build
the Location: response header. We planned to implement that at one
moment but it's still not done because there was no use of it yet.
Do you have a lot of domains in your wildcard certificate ? In a first
time you could write as many redirect rules as you have domains, if
that's not too many. For instance :

  acl name1 hdr(host) -i name1.domain.com
  acl name2 hdr(host) -i name2.domain.com
  acl name3 hdr(host) -i name3.domain.com
  redirect prefix https://name1.domain.com if name1
  redirect prefix https://name2.domain.com if name2
  redirect prefix https://name3.domain.com if name3

See ?


Using haproxy 1.3.19 on a Centos 5 machine.


Be careful with 1.3.19, it has a bug causing a random crash if you
forget to set some timeouts on some instances. So be careful to
have a defaults section with at least client/server/connect timeouts
set.

Regards,
Willy

Re: Redirect_prefix question

[Willy Tarreau]

On Sat, Oct 17, 2009 at 04:06:30PM +0100, Chris Sarginson wrote:
 Unfortunately there are quite a few subdomains, and we don't know them  
 all as this is for a client, who will just be looking to add dns  
 records and vhosts on the fly. What sort of work is required to add in  
 this kind of functionality?

Either we decide to start with the complex method, something like :

redirect prefix http://$hdr(host) ...

Or we decide that http/https with one header is about the only one we're
interested in for this type of operation, and we implement :

redirect prefix_http  host ...
redirect prefix_https host ...

The first one will set http://; followed by whatever lies in the host
field. The second one will do the same with https://;.

In my opinion we still need to have the header name configurable
because behind some reverse proxies (eg: apache), the host header
will be rewritten and we need to use x-forwarded-host.

I don't know if anyone is willing to take that task.

Regards,
Willy