Re: BADREQ on production haproxy

2012-05-12 Thread Willy Tarreau
Hi Bar, On Thu, May 10, 2012 at 07:02:58PM +0300, Bar Ziony wrote: Hey, We're running haproxy 1.4.20 as our LB, nginx is listening on the same machine on port 443 and terminating SSL, proxying the unencrypted requests to haproxy on localhost:80. I see many of these errors on the haproxy

Re: How can I benchmark the 'source' balance scheme

2012-05-12 Thread Willy Tarreau
Hi, On Fri, May 11, 2012 at 05:45:26PM +0200, Joeri Blokhuis | DongIT wrote: On 05/11/2012 04:50 PM, Baptiste wrote: On Fri, May 11, 2012 at 4:36 PM, Joeri Blokhuis | DongIT joeri.blokh...@dongit.nl wrote: Hello guys, I would like to benchmark and test the 'source' balance algorithm of

Re: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
Hi, On Fri, May 11, 2012 at 10:32:33PM +0200, Baptiste wrote: On Fri, May 11, 2012 at 6:09 PM, Andres Thomas Stivalet atstiva...@gmail.com wrote: Currently, if haproxy tries to start up and a webserver is down (and it's defined as a hostname in the config file) haproxy refuses to start.

Re: unresolvable host names as error

2012-05-12 Thread Baptiste
Well, I must say I'm unsure what we'll do upon startup, because if we accept to start with buggy host names, a number of admins will get trapped with errors in their configs causing servers to remain down forever. At least right now they know from the beginning that they have to fix their

Re: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 11:09:22AM +0200, Baptiste wrote: Well, I must say I'm unsure what we'll do upon startup, because if we accept to start with buggy host names, a number of admins will get trapped with errors in their configs causing servers to remain down forever. At least right now

Re: BADREQ on production haproxy

2012-05-12 Thread Bar Ziony
Willy, Thank you, I will follow up with your suggestions soon. But I just had a production down-time with the haproxy machine: After posting something to our Facebook wall (it happened twice, yesterday and 3 days ago), which usually brings more traffic (but not more than we can usually handle

Re: BADREQ on production haproxy

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 01:23:17PM +0200, Baptiste wrote: On Sat, May 12, 2012 at 1:01 PM, Bar Ziony bar...@gmail.com wrote: Willy, Thank you, I will follow up with your suggestions soon. But I just had a production down-time with the haproxy machine: After posting something to our

Fwd: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
Sometimes it would be resolvable and other times not, it depends if the host has reported itself to the DNS server previously and it's still in the bind cache after going down. I suspect that this is happening if the host is down and there is no IP for it in the DNS tables, but that's just guess

Re: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
Hi Willy, I believe in previous versions of haproxy having an unresolvable host name at start up did not cause haproxy to error out. Bellow is the change log I found: 2010/03/30 : 1.3.24 - [BUG] config: report unresolvable host names as errors I wrongly assumed that haproxy performed DNS

Re: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 10:25:58AM -0500, Andres Thomas Stivalet wrote: Hi Willy, I believe in previous versions of haproxy having an unresolvable host name at start up did not cause haproxy to error out. Bellow is the change log I found: 2010/03/30 : 1.3.24 - [BUG] config: report

Re: BADREQ on production haproxy

2012-05-12 Thread Bar Ziony
I have no problem increasing the RAM if needed, but how do I know if it's needed? Where can I see the number of connections per second to see if I somehow reached 20k ? I don't think I reached 20k because the global maxconn is 20K This is my TCP tuning config for the LB: # TCP stack tuning

Stats for backend queue

2012-05-12 Thread Bar Ziony
Hey, I have a dynamic backend with maxconn 80 with multiple servers. Many times I can see on the haproxy stats page that servers on this backend are reaching their maximum 80, but I don't see the number of requests currently in queue. The maximum number I ever see is 80. Why is that? Can I

Re: BADREQ on production haproxy

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 06:54:06PM +0300, Bar Ziony wrote: I have no problem increasing the RAM if needed, but how do I know if it's needed? Where can I see the number of connections per second to see if I somehow reached 20k ? I have not said 20k conns/s, but 20k concurrent conns. Concurrency

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 07:01:19PM +0300, Bar Ziony wrote: Hey, I have a dynamic backend with maxconn 80 with multiple servers. Many times I can see on the haproxy stats page that servers on this backend are reaching their maximum 80, but I don't see the number of requests currently in

Re: Stats for backend queue

2012-05-12 Thread Baptiste
On Sat, May 12, 2012 at 6:01 PM, Bar Ziony bar...@gmail.com wrote: Hey, I have a dynamic backend with maxconn 80 with multiple servers. Many times I can see on the haproxy stats page that servers on this backend are reaching their maximum 80, but I don't see the number of requests currently

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Willy, thanks for your answer. On Sat, May 12, 2012 at 7:21 PM, Willy Tarreau w...@1wt.eu wrote: On Sat, May 12, 2012 at 07:01:19PM +0300, Bar Ziony wrote: Hey, I have a dynamic backend with maxconn 80 with multiple servers. Many times I can see on the haproxy stats page that servers

Re: BADREQ on production haproxy

2012-05-12 Thread Bar Ziony
Hi Willy, On Sat, May 12, 2012 at 7:08 PM, Willy Tarreau w...@1wt.eu wrote: On Sat, May 12, 2012 at 06:54:06PM +0300, Bar Ziony wrote: I have no problem increasing the RAM if needed, but how do I know if it's needed? Where can I see the number of connections per second to see if I

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 08:43:43PM +0300, Bar Ziony wrote: So session rate is the number of requests per second ? Why is it called session then if it's really requests? You have the two. Initially in haproxy, you had no keepalive, so 1 req = 1 session. Now you have the numbers in the session

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Hi Willy :) On Sat, May 12, 2012 at 10:06 PM, Willy Tarreau w...@1wt.eu wrote: On Sat, May 12, 2012 at 08:43:43PM +0300, Bar Ziony wrote: So session rate is the number of requests per second ? Why is it called session then if it's really requests? You have the two. Initially in haproxy,

Re: Stats for backend queue

2012-05-12 Thread Cyril Bonté
Hi, Le 12/05/2012 21:42, Bar Ziony a écrit : OK, got why there are more frontend sessions than backend sessions. But is it usual to see so much more? In the configuration you provided, you didn't set any timeout http-keep-alive. It means that on your frontend, your keep-alive timeout is

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Oh, thanks. Small value = 10 sec for example? :| What is an optimal keepalive timeout? Thanks, Bar. On Sat, May 12, 2012 at 10:51 PM, Cyril Bonté cyril.bo...@free.fr wrote: Hi, Le 12/05/2012 21:42, Bar Ziony a écrit : OK, got why there are more frontend sessions than backend sessions. But

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 02:53:45PM -0500, Andres Thomas Stivalet wrote: That's why I've added this to the TODO list :-) Thanks Willy! For the same reason, you cannot safely install firewalls nor VPNs in such environments. I'd just like to mention that it's certainly possible to

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 11:04:49PM +0300, Bar Ziony wrote: Oh, thanks. Small value = 10 sec for example? :| What is an optimal keepalive timeout? I like to use just a few seconds so that all objects from the same page are fetched at once and the connection automatically closes after this. But

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Is there a benefit to allow a larger keepalive timeout so more than resources from 1 page will be downloaded, or is it just best to create a new connection for succeeding pages? Willy, did you see my previous email in this correspondence? :) Thanks, Bar. On Sat, May 12, 2012 at 11:18 PM, Willy

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 10:42:24PM +0300, Bar Ziony wrote: How can I know the average response time of my servers? haproxy provides that data somewhere? Yes you have each response time value in your logs :-) But can I get the average response time? By doing an average of what appears

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 11:31:03PM +0300, Bar Ziony wrote: Is there a benefit to allow a larger keepalive timeout so more than resources from 1 page will be downloaded, or is it just best to create a new connection for succeeding pages? It depends on your available memory. Ideally you'd keep

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
Given how the current farm toolset operates under the hostname assumption we're still going to have change hostnames etc. We're currently evaluating if we want to patch haproxy for our needs or track our machines some other way. Patching haproxy seems like the sane route. :) On May 12, 2012 3:17

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Thank you Willy! I increased the RAM to 2GB and now I don't see the problem. I will change the buffers size and report back as well. Why did you recommend bufsize of 8130 and not 8192? Also, Why do I need to change tune.maxrewrite ? Thanks, Bar. On Sat, May 12, 2012 at 11:42 PM, Willy Tarreau

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sun, May 13, 2012 at 12:50:39AM +0300, Bar Ziony wrote: Thank you Willy! I increased the RAM to 2GB and now I don't see the problem. I will change the buffers size and report back as well. Why did you recommend bufsize of 8130 and not 8192? As I explained in an earlier mail in this