Re: How does balance leastconn work with long-lived HTTP sessions?
On Mon, Aug 11, 2014 at 1:34 PM, Lukas Tribus luky...@hotmail.com wrote: I would say this simple depends on the http mode you use. When you use tunnel mode (default in pre-1.5 releases), it obviously sticks to the same backend server. When you use close mode, both server and client connections are closed after the response (no keepalive). Instead, for your environment you should use http-server-close: http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option%20http-server-close Does this actually close the HTTP session, close the sockets, etc? Because of the large number of requests per second (~150k over our whole cluster) I'm afraid that the overhead of connection establishment might be problematic for us if so. This way, frontend keeps doing keepalive with your clients, maintaining long-lived sessions, while your backend closes the connection after the response, thus, leastconn works for every single requests. Is there a hybrid mode, where connections (in the TCP sense) are kept alive between HAProxy and the backends, but where individual HTTP requests are potentially balanced to any of the existing backend connections? You mentioned pre-1.5 (we're on 1.4.x), has something changed in 1.5 that might be a fit for our use case? Thanks, - Dan
Re: What is hrsp_5xx for a frontend?
Thanks, Willy, that makes sense. What settings should we look at tuning? We already have backlog set to 32k in the defaults, but with fairly low timeouts for connect, queue, and server. Should we try setting those timeouts somewhat higher? Or would you recommend something else? (I can also post the haproxy.cfg if that will help) MAGNE+IC Dan Crosta | Director, Engineering On Mon, May 26, 2014 at 10:17 AM, Willy Tarreau w...@1wt.eu wrote: On Sat, May 24, 2014 at 08:40:35AM -0400, Dan Crosta wrote: I'm a little confused about what exactly creates a 5xx response code from HAProxy. I think that if the backend generates a 5xx response, this will show up as a frontend_5xx, and I expect that the value of the hrsp_5xx stat for the frontend should equal the sum of the hrsp_5xx values from the backends. However, this doesn't jive with what we actually see in the statistics output: $ curl -s http://10.1.99.106:8080/haproxy?stats;csv;norefresh; | tr , | cut -f 1,2,44 | head -n 19 # pxname svnamehrsp_5xx myfrontendFRONTEND 10797124 mybackend host001 24364 mybackend host002 17260 mybackend host003 25250 mybackend host004 15135 mybackend host005 17364 mybackend host006 24541 mybackend host007 11422 mybackend host008 14199 mybackend host009 13274 mybackend host010 12797 mybackend host011 24532 mybackend host012 23599 mybackend host013 31728 mybackend host014 24607 mybackend host015 14851 mybackend host016 13937 mybackend BACKEND 10797124 The sum of the hrsp_5xx's for each of the individual backends is 308860, while both the FRONTEND and BACKEND numbers are about 30 times as much. Am I missing something obvious? Does HAProxy ever generate 5xx responses itself, either in the backend or frontend? Is there some more detailed logging we should enable to track this down? Yes it can generate 5xx when it cannot deliver the request somewhere. For example if you're running with many use_backend rules and no default_backend, it's possible that a number of requests en up going nowhere and are rejected as 503. In your case since we're seeing the same number in the frontend and in the backend and you have much less errors on the servers themselves, that can be explained by 503 when the queue is overflown (requests that are never served because of too low a server maxconn value, too short queue timeouts, or too long server response times). Hoping this helps, Willy
What is hrsp_5xx for a frontend?
I'm a little confused about what exactly creates a 5xx response code from HAProxy. I think that if the backend generates a 5xx response, this will show up as a frontend_5xx, and I expect that the value of the hrsp_5xx stat for the frontend should equal the sum of the hrsp_5xx values from the backends. However, this doesn't jive with what we actually see in the statistics output: $ curl -s http://10.1.99.106:8080/haproxy?stats;csv;norefresh; | tr , | cut -f 1,2,44 | head -n 19 # pxname svnamehrsp_5xx myfrontendFRONTEND 10797124 mybackend host001 24364 mybackend host002 17260 mybackend host003 25250 mybackend host004 15135 mybackend host005 17364 mybackend host006 24541 mybackend host007 11422 mybackend host008 14199 mybackend host009 13274 mybackend host010 12797 mybackend host011 24532 mybackend host012 23599 mybackend host013 31728 mybackend host014 24607 mybackend host015 14851 mybackend host016 13937 mybackend BACKEND 10797124 The sum of the hrsp_5xx's for each of the individual backends is 308860, while both the FRONTEND and BACKEND numbers are about 30 times as much. Am I missing something obvious? Does HAProxy ever generate 5xx responses itself, either in the backend or frontend? Is there some more detailed logging we should enable to track this down? Thanks, - Dan
Re: What is hrsp_5xx for a frontend?
(Oops, and apologies for the HTML formatting on that last one)
