Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sun, May 13, 2012 at 12:50:39AM +0300, Bar Ziony wrote: > Thank you Willy! > > I increased the RAM to 2GB and now I don't see the problem. I will change > the buffers size and report back as well. > Why did you recommend bufsize of 8130 and not 8192? As I explained in an earlier mail in this t

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 03:52:22PM -0500, Andres Thomas Stivalet wrote: > Given how the current farm toolset operates under the hostname assumption > we're still going to have change hostnames etc. > > We're currently evaluating if we want to patch haproxy for our needs or > track our machines som

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Thank you Willy! I increased the RAM to 2GB and now I don't see the problem. I will change the buffers size and report back as well. Why did you recommend bufsize of 8130 and not 8192? Also, Why do I need to change tune.maxrewrite ? Thanks, Bar. On Sat, May 12, 2012 at 11:42 PM, Willy Tarreau w

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
Given how the current farm toolset operates under the hostname assumption we're still going to have change hostnames etc. We're currently evaluating if we want to patch haproxy for our needs or track our machines some other way. Patching haproxy seems like the sane route. :) On May 12, 2012 3:17

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 11:31:03PM +0300, Bar Ziony wrote: > Is there a benefit to allow a larger keepalive timeout so more than > resources from 1 page will be downloaded, or is it just best to create a > new connection for succeeding pages? It depends on your available memory. Ideally you'd keep

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 10:42:24PM +0300, Bar Ziony wrote: > > > How can I know the average response time of my servers? haproxy provides > > > that data somewhere? > > > > Yes you have each response time value in your logs :-) > > But can I get the average response time? By doing an average of w

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Is there a benefit to allow a larger keepalive timeout so more than resources from 1 page will be downloaded, or is it just best to create a new connection for succeeding pages? Willy, did you see my previous email in this correspondence? :) Thanks, Bar. On Sat, May 12, 2012 at 11:18 PM, Willy T

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 11:04:49PM +0300, Bar Ziony wrote: > Oh, thanks. > Small value = 10 sec for example? :| What is an optimal keepalive timeout? I like to use just a few seconds so that all objects from the same page are fetched at once and the connection automatically closes after this. But

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 02:53:45PM -0500, Andres Thomas Stivalet wrote: > >That's why I've added this to the TODO list :-) > > Thanks Willy! > > > For the same reason, you cannot safely install firewalls nor VPNs in such > > environments. > > I'd just like to mention that it's certainly possibl

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Oh, thanks. Small value = 10 sec for example? :| What is an optimal keepalive timeout? Thanks, Bar. On Sat, May 12, 2012 at 10:51 PM, Cyril Bonté wrote: > Hi, > > Le 12/05/2012 21:42, Bar Ziony a écrit : > > OK, got why there are more frontend sessions than backend sessions. But >> is it usual

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
>That's why I've added this to the TODO list :-) Thanks Willy! > For the same reason, you cannot safely install firewalls nor VPNs in such > environments. I'd just like to mention that it's certainly possible to implement a safe dynamic dhcp+dns environment for a private network. Just to give y

Re: Stats for backend queue

2012-05-12 Thread Cyril Bonté
Hi, Le 12/05/2012 21:42, Bar Ziony a écrit : OK, got why there are more frontend sessions than backend sessions. But is it usual to see so much more? In the configuration you provided, you didn't set any "timeout http-keep-alive". It means that on your frontend, your keep-alive timeout is eq

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Hi Willy :) On Sat, May 12, 2012 at 10:06 PM, Willy Tarreau wrote: > On Sat, May 12, 2012 at 08:43:43PM +0300, Bar Ziony wrote: > > So session rate is the number of requests per second ? Why is it called > > session then if it's really requests? > > You have the two. Initially in haproxy, you ha

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 08:43:43PM +0300, Bar Ziony wrote: > So session rate is the number of requests per second ? Why is it called > session then if it's really requests? You have the two. Initially in haproxy, you had no keepalive, so 1 req = 1 session. Now you have the numbers in the session c

Re: BADREQ on production haproxy

2012-05-12 Thread Bar Ziony
Hi Willy, On Sat, May 12, 2012 at 7:08 PM, Willy Tarreau wrote: > On Sat, May 12, 2012 at 06:54:06PM +0300, Bar Ziony wrote: > > I have no problem increasing the RAM if needed, but how do I know if it's > > needed? Where can I see the number of connections per second to see if I > > somehow rea

Re: Stats for backend queue

2012-05-12 Thread Bar Ziony
Willy, thanks for your answer. On Sat, May 12, 2012 at 7:21 PM, Willy Tarreau wrote: > On Sat, May 12, 2012 at 07:01:19PM +0300, Bar Ziony wrote: > > Hey, > > > > I have a dynamic backend with maxconn 80 with multiple servers. > > Many times I can see on the haproxy stats page that servers on t

Re: Stats for backend queue

2012-05-12 Thread Baptiste
On Sat, May 12, 2012 at 6:01 PM, Bar Ziony wrote: > Hey, > > I have a dynamic backend with maxconn 80 with multiple servers. > Many times I can see on the haproxy stats page that servers on this backend > are reaching their maximum 80, but I don't see the number of requests > currently in queue. T

Re: Stats for backend queue

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 07:01:19PM +0300, Bar Ziony wrote: > Hey, > > I have a dynamic backend with maxconn 80 with multiple servers. > Many times I can see on the haproxy stats page that servers on this backend > are reaching their maximum 80, but I don't see the number of requests > currently in

Re: BADREQ on production haproxy

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 06:54:06PM +0300, Bar Ziony wrote: > I have no problem increasing the RAM if needed, but how do I know if it's > needed? Where can I see the number of connections per second to see if I > somehow reached 20k ? I have not said 20k conns/s, but 20k concurrent conns. Concurren

Stats for backend queue

2012-05-12 Thread Bar Ziony
Hey, I have a dynamic backend with maxconn 80 with multiple servers. Many times I can see on the haproxy stats page that servers on this backend are reaching their maximum 80, but I don't see the number of requests currently in queue. The maximum number I ever see is 80. Why is that? Can I somehow

Re: BADREQ on production haproxy

2012-05-12 Thread Bar Ziony
I have no problem increasing the RAM if needed, but how do I know if it's needed? Where can I see the number of connections per second to see if I somehow reached 20k ? I don't think I reached 20k because the global maxconn is 20K This is my TCP tuning config for the LB: # TCP stack tuning net

Re: Fwd: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 10:24:16AM -0500, Andres Thomas Stivalet wrote: > The name lookup during health checks + not erroring out on start-up, > seems like this would correct the problem. Currently, it seems haproxy > is rather unsafe to use for a dynamic dhcp+dns farm setup. This is clearly true.

Re: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 10:25:58AM -0500, Andres Thomas Stivalet wrote: > Hi Willy, > > I believe in previous versions of haproxy having an unresolvable host > name at start up did not cause haproxy to error out. Bellow is the > change log I found: > > 2010/03/30 : 1.3.24 > - [BUG] config: re

Re: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
Hi Willy, I believe in previous versions of haproxy having an unresolvable host name at start up did not cause haproxy to error out. Bellow is the change log I found: 2010/03/30 : 1.3.24 - [BUG] config: report unresolvable host names as errors I wrongly assumed that haproxy performed DNS loo

Fwd: unresolvable host names as error

2012-05-12 Thread Andres Thomas Stivalet
Sometimes it would be resolvable and other times not, it depends if the host has reported itself to the DNS server previously and it's still in the bind cache after going down. I suspect that this is happening if the host is down and there is no IP for it in the DNS tables, but that's just guess wo

Re: BADREQ on production haproxy

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 01:23:17PM +0200, Baptiste wrote: > On Sat, May 12, 2012 at 1:01 PM, Bar Ziony wrote: > > Willy, > > > > Thank you, I will follow up with your suggestions soon. > > > > But I just had a production down-time with the haproxy machine: > > After posting something to our Facebo

Re: BADREQ on production haproxy

2012-05-12 Thread Baptiste
On Sat, May 12, 2012 at 1:01 PM, Bar Ziony wrote: > Willy, > > Thank you, I will follow up with your suggestions soon. > > But I just had a production down-time with the haproxy machine: > After posting something to our Facebook wall (it happened twice, yesterday > and 3 days ago), which usually b

Re: BADREQ on production haproxy

2012-05-12 Thread Bar Ziony
Willy, Thank you, I will follow up with your suggestions soon. But I just had a production down-time with the haproxy machine: After posting something to our Facebook wall (it happened twice, yesterday and 3 days ago), which usually brings more traffic (but not more than we can usually handle (fo

Re: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
On Sat, May 12, 2012 at 11:09:22AM +0200, Baptiste wrote: > > Well, I must say I'm unsure what we'll do upon startup, because if we > > accept to start with buggy host names, a number of admins will get > > trapped with errors in their configs causing servers to remain down > > forever. At least ri

Re: unresolvable host names as error

2012-05-12 Thread Baptiste
> Well, I must say I'm unsure what we'll do upon startup, because if we > accept to start with buggy host names, a number of admins will get > trapped with errors in their configs causing servers to remain down > forever. At least right now they know from the beginning that they > have to fix their

Re: unresolvable host names as error

2012-05-12 Thread Willy Tarreau
Hi, On Fri, May 11, 2012 at 10:32:33PM +0200, Baptiste wrote: > On Fri, May 11, 2012 at 6:09 PM, Andres Thomas Stivalet > wrote: > > Currently, if haproxy tries to start up and a webserver is down (and > > it's defined as a hostname in the config file) haproxy refuses to > > start. Looking at the

Re: How can I benchmark the 'source' balance scheme

2012-05-12 Thread Willy Tarreau
Hi, On Fri, May 11, 2012 at 05:45:26PM +0200, Joeri Blokhuis | DongIT wrote: > On 05/11/2012 04:50 PM, Baptiste wrote: > > On Fri, May 11, 2012 at 4:36 PM, Joeri Blokhuis | DongIT > > wrote: > >> Hello guys, > >> > >> I would like to benchmark and test the 'source' balance algorithm of > >> HAPro

Re: BADREQ on production haproxy

2012-05-12 Thread Willy Tarreau
Hi Bar, On Thu, May 10, 2012 at 07:02:58PM +0300, Bar Ziony wrote: > Hey, > > We're running haproxy 1.4.20 as our LB, nginx is listening on the same > machine on port 443 and terminating SSL, proxying the unencrypted requests > to haproxy on localhost:80. > > I see many of these errors on the ha