Re: Error during Apache AB benchmarking test -- HAProxy
Thanks Willy...will give that a try and let you know how it goes --- On Fri, 2/4/11, Willy Tarreau wrote: From: Willy Tarreau Subject: Re: Error during Apache AB benchmarking test -- HAProxy To: "Amol" Cc: "carlo flores" , haproxy@formilux.org Date: Friday, February 4, 2011, 3:16 PM On Thu, Feb 03, 2011 at 09:50:56PM -0800, Amol wrote: > I actually see the same error while going to the individual servers as well > and i am using apache server > Server version: Apache/2.2.14 (Ubuntu) Check if ip_conntrack or nf_conntrack is loaded on your server during your tests. If so, your might have filled the connection table (check last dmesg entries for this). Willy
Re: Configuration API?
Try using Hatop...this will do all the things from your list http://feurix.org/projects/hatop/ I am using it, and learning more about it now.. --- On Fri, 2/4/11, Joel Krauska wrote: From: Joel Krauska Subject: Re: Configuration API? To: "John Marrett" Cc: haproxy@formilux.org Date: Friday, February 4, 2011, 6:04 PM Awesome - This is a great start. Putting a server in backup or primary mode would be a nice addition. Adding new servers would also be nice, but that's likely to be a harder project. --Joel On 2/4/11 12:46 PM, John Marrett wrote: > Check out section "9.2. Unix Socket commands" > > http://haproxy.1wt.eu/download/1.4/doc/configuration.txt > > -JohnF > > On 11-02-04 03:40 PM, Joel Krauska wrote: >> Has there ever been effort in building out a configuration API for >> HAProxy? >> >> It would be nice to be able to tweak configs externally to do things. >> (simpler things now, but obviously being able to tweak /ANY/ config >> parameter might be nice down the road) >> >> My wish list: >> - put a server in backup mode or primary node as needed >> - raise or lower a server weight >> - add a new server to an existing pool >> >> How do other folks do this in a programmatic fashion? >> (just adjust config files and HUP?) >> >> --Joel >> >
Re: Configuration API?
Awesome - This is a great start. Putting a server in backup or primary mode would be a nice addition. Adding new servers would also be nice, but that's likely to be a harder project. --Joel On 2/4/11 12:46 PM, John Marrett wrote: Check out section "9.2. Unix Socket commands" http://haproxy.1wt.eu/download/1.4/doc/configuration.txt -JohnF On 11-02-04 03:40 PM, Joel Krauska wrote: Has there ever been effort in building out a configuration API for HAProxy? It would be nice to be able to tweak configs externally to do things. (simpler things now, but obviously being able to tweak /ANY/ config parameter might be nice down the road) My wish list: - put a server in backup mode or primary node as needed - raise or lower a server weight - add a new server to an existing pool How do other folks do this in a programmatic fashion? (just adjust config files and HUP?) --Joel
Win Rs. 50,000 this feb!
To stop further mailings, click here. To report spam, click here. YouMint treats your privacy VERY SERIOUSLY and will take strict action against the sender if you don't know them personally. Win Rs 50,000 in Cash (Tax Free) in February 2011!Hey,YouMint is celebrating 50 Lakh members! The Big Bash is live now!- Win Rs 50,000 in Feb - Totally Tax Free! - Win Rs 1000 daily in February just for checking-in on YouMint!- Get Free Mobile Recharge on most operators in India!- Get upto Rs 100 for every friend wh o joins 'The Legaue'! Go wild inviting!Create your YouMint account now and rake it in big this month.Join Here: http://youmint.com/bigbash-replyskumar CheersSanjeev Kumar Copyright 2007-2010 YouMint Media (P) Ltd.
Re: HAproxy Tuning - HTTP requests loading serially
Hi David, On Wed, Feb 02, 2011 at 05:42:20PM -0800, David Tosoff wrote: > Thanks Peres, > > I had tried that already. serv-close actually took my sites and games down to > a > crawl. > > I tried pretend keep-alives as well and removing httpclose all together; same > result; seems best with httpclose enabled. > > Any other Ideas/suggestions? Before 1.4.9, combining http-server-close with pretend-keepalive could result in what you observed because the server sometimes did not send a close in the response, and the client thought it wanted to stay alive. But some servers also failed to send a content-length or transfer-encoding if they got a close, resulting in a stupid situation where the client has to wait for the server to actively close to detect the end. 1.4.9 fixed that but introduced a new similar issue when combining httpclose with pretend-keepalive. All those are fixed in latest -git which will soon be released as 1.4.11. Anyway, 1.4.10 with http-server-close + http-keepalive SHOULD be OK. Please tell us if you still get the issue with that version. > On another note, I notice that haproxy strips out/uncompresses any gzipping > the > server replies with. Could this be related? Yes, though it's not haproxy which strips it, it's that there was a bug in at least one server (tomcat) which decided that an HTTP/1.1 request with "Connection: close" was equivalent to an HTTP/1.0 request (which it is not). So due to this, it refrained from using transfer encoding which is needed to send compressed contents. This bug was reported to the tomcat team and fixed in latest version if I understood it right. This is what led us to implement the "pretend-keepalive" option. It's possible that other servers have the same bug though I don't have a list. Cheers, Willy
Re: Configuration API?
As an example, our shop extends your wish list with the ability to pull stats from the UNIX socket and trigger alerts via Nagios or Cloudkick ass we need. If you want it, easy to add whatever you need to a fork of this. We'll commit it back in: https://github.com/flores/haproxyctl I believe you can only enable or disable already configured servers in your pool. A way around this could be to set disabled servers to some floating ip address/dns you assign to new servers. Still limited by the number of preconfigured ips/dns records of course. Maybe there's a better way... On Fri, Feb 4, 2011 at 12:46 PM, John Marrett wrote: > Check out section "9.2. Unix Socket commands" > > http://haproxy.1wt.eu/download/1.4/doc/configuration.txt > > -JohnF > > > On 11-02-04 03:40 PM, Joel Krauska wrote: > >> Has there ever been effort in building out a configuration API for >> HAProxy? >> >> It would be nice to be able to tweak configs externally to do things. >> (simpler things now, but obviously being able to tweak /ANY/ config >> parameter might be nice down the road) >> >> My wish list: >> - put a server in backup mode or primary node as needed >> - raise or lower a server weight >> - add a new server to an existing pool >> >> How do other folks do this in a programmatic fashion? >> (just adjust config files and HUP?) >> >> --Joel >> >> > >
Re: balance (hdr) problem (maybe bug?)
Hi Craig, On Thu, Feb 03, 2011 at 02:59:40PM +0100, Craig Craig wrote: > Hi, > > I've stumbled upon a problem with balance(hdr), specefically with > X-Forwarded-For. > When you use the config that I've attached, you get different results > wheather you send a X-Forwarded-For or not. > > The source IP does not change when I perform those queries, hosts did not > change state: > > curl http://www.foo.de/host.jsp -s > Stays always on the same server. > > curl http://www.foo.de/host.jsp -s -H "X-Forwarded-For: x.x.x.x" > Jumps between the three hosts. > > This is strange: I delete the header that is sent by the client on the > frontend with reqidel and set a new one with "option forwardfor" - I expected > the backend to balance based on that new header. > > If my assumption was wrong, and the original header is used, then I should > not jump between hosts when I am always sending the same header. > > Something smells fishy here...is this a bug? A Feature? ;) Or > misunderstanding on my part? It's not how it's supposed to work, but I agree the doc is not very precise about that. The X-Forwarded-For header is only added once at the end of all processing. Otherwise, having it in the defaults section would result in both your frontend and your backend adding it. So in your case, what happens is that you delete it in the frontend (using reqidel) then you tag the session for adding a new one after all processing is done. When at the last point we have to establish a connection to the server, we check the header and balance based on it. I agree we should always have it filled with the same value, so there's a bug. My guess is that you're running a version prior to 1.4.10 which has the header deletion bug : the header list can become corrupted when exactly two consecutive headers are removed from the request (eg: connection and x-forwarded-for). Then the newly added X-Forwarded-For could not be seen by the code responsible for hashing it. If so, please try to upgrade to the last bug fix (1.4.10) and see if the problem persists. Also, I'd like to add that what you're doing is simply equivalent (though more complex) to hashing the source address. You'd better use "balance src" for this :-) Cheers, Willy
Re: Configuration API?
Check out section "9.2. Unix Socket commands" http://haproxy.1wt.eu/download/1.4/doc/configuration.txt -JohnF On 11-02-04 03:40 PM, Joel Krauska wrote: Has there ever been effort in building out a configuration API for HAProxy? It would be nice to be able to tweak configs externally to do things. (simpler things now, but obviously being able to tweak /ANY/ config parameter might be nice down the road) My wish list: - put a server in backup mode or primary node as needed - raise or lower a server weight - add a new server to an existing pool How do other folks do this in a programmatic fashion? (just adjust config files and HUP?) --Joel
Configuration API?
Has there ever been effort in building out a configuration API for HAProxy? It would be nice to be able to tweak configs externally to do things. (simpler things now, but obviously being able to tweak /ANY/ config parameter might be nice down the road) My wish list: - put a server in backup mode or primary node as needed - raise or lower a server weight - add a new server to an existing pool How do other folks do this in a programmatic fashion? (just adjust config files and HUP?) --Joel
Re: filtering HTTP HEAD requests
Hi Jan-Frode, On Fri, Feb 04, 2011 at 03:15:49PM +0100, Jan-Frode Myklebust wrote: > We're trying to filter out HTTP HEAD requests using haproxy, > to make the proxy behave like shoutcast servers, ref: > > http://comments.gmane.org/gmane.comp.web.haproxy/2961 But why the hell are your trying to deliberately break HTTP ? It might be a lot better for the long term to fix the server instead, or you'll constantly have to resort to dirty tricks to achieve that. >From RFC2616 par.5.1.2: The methods GET and HEAD MUST be supported by all general-purpose servers. All other methods are OPTIONAL. > We're using haproxy-1.4.8-1.el6.x86_64 (EPEL) and have configured: > > reqdeny ^HEAD > errorfile 403 /dev/null > > This was working, but now it seems our clients have changed behaviour, > and it doesn't work anymore :-( With this configuration haproxy is > close to, but not 100% replicating the behaviour of shoutcast servers. > HAproxy waits for an empty line before aborting, while shoutcast > servers abort immediately when they're sent the HEAD request: > > haproxy: > -- > $ telnet tvproxy.example.com 80 > Trying my.ip.here... > Connected to tvproxy.example.com. > Escape character is '^]'. > HEAD /webradioproxy/8000/74.63.99.170/ HTTP/1.0 > > Connection closed by foreign host. > -- > > shoutcast: > -- > $ telnet 74.63.99.170 8000 > Trying 74.63.99.170... > Connected to 74.63.99.170. > Escape character is '^]'. > HEAD / HTTP/1.0 > Connection closed by foreign host. > -- > > Is there any way to get haproxy to abort before empty line also ? No, because HTTP request is complete only once the empty line is received, and processing only happens on complete requests (otherwise you couldn't use ACLs nor anything). In pure TCP in theory it would be possible, but we still don't have any pattern extraction method to match on arbitrary data. And it would not work on keep-alive requests either anyway. I'd say that the fact that your client behaviour have changed to something even dirtier might be an indication that what you're doing is wrong. What you can do though is to quickly reject incomplete requests. Use a very low "timeout http-request" (eg: 1s). If they fail to send the whole request at once, they'll get a "HTTP/1.0 408 Request Timeout". At least with this, the clients' users will notice the issue and report it to the author. Similarly, for the HEAD, you should return a "501 Method not implemented", it would be much cleaner to educate your clients than closing the connection, because they'll be able to diagnose the issue. Regards, Willy
Re: Adding backend server as a response header?
Hi,
On Tue, Feb 01, 2011 at 04:08:43PM -0500, Chase Seibert wrote:
> In this example config, is it possible to insert a custom response
> header, say "X-Backend-Server", with the value of "www1" or "www2"
> depending on which server HAProxy routed to? This would be primarily
> for trouble-shooting purposes. For example, if we suspect that the
> deploy is broken on just one of the web servers.
In very recent version (the yet unreleased 1.4.11), you can do that
with the "srv_id" acl, which allows you to match an ACL based on the
server which processed the request. Thus you can have :
rspadd X-Backend-Server:\ www1 if { srv_id 1 }
rspadd X-Backend-Server:\ www2 if { srv_id 2 }
Obviously this will not scale very well but can be useful for trouble
shooting purposes.
Alternatively, you can simply insert a cookie in the response the
usual way, but it will enforce persistence. I've checked and reponse
rewriting is performed before sending persitence cookie so it's not
possible to use a reqdel rule to delete the added cookie. You might
delete all client cookies if it's a static server though.
I've already got demands for adding the server name in requests
(which is very difficult due to redispatches), but I'm realizing
it would be much easier in responses (though it does not serve the
same purpose). Maybe we should simply add an option for that, it
would be just a few lines of code.
Regards,
Willy
Re: Error during Apache AB benchmarking test -- HAProxy
On Thu, Feb 03, 2011 at 09:50:56PM -0800, Amol wrote: > I actually see the same error while going to the individual servers as well > and i am using apache server > Server version: Apache/2.2.14 (Ubuntu) Check if ip_conntrack or nf_conntrack is loaded on your server during your tests. If so, your might have filled the connection table (check last dmesg entries for this). Willy
filtering HTTP HEAD requests
We're trying to filter out HTTP HEAD requests using haproxy, to make the proxy behave like shoutcast servers, ref: http://comments.gmane.org/gmane.comp.web.haproxy/2961 We're using haproxy-1.4.8-1.el6.x86_64 (EPEL) and have configured: reqdeny ^HEAD errorfile 403 /dev/null This was working, but now it seems our clients have changed behaviour, and it doesn't work anymore :-( With this configuration haproxy is close to, but not 100% replicating the behaviour of shoutcast servers. HAproxy waits for an empty line before aborting, while shoutcast servers abort immediately when they're sent the HEAD request: haproxy: -- $ telnet tvproxy.example.com 80 Trying my.ip.here... Connected to tvproxy.example.com. Escape character is '^]'. HEAD /webradioproxy/8000/74.63.99.170/ HTTP/1.0 Connection closed by foreign host. -- shoutcast: -- $ telnet 74.63.99.170 8000 Trying 74.63.99.170... Connected to 74.63.99.170. Escape character is '^]'. HEAD / HTTP/1.0 Connection closed by foreign host. -- Is there any way to get haproxy to abort before empty line also ? -jf
