Thanks for all your help Baptiste, I'll keep poking. :)
On Wed, May 28, 2014 at 11:02 AM, Baptiste <[email protected]> wrote: > On Wed, May 28, 2014 at 4:47 PM, Jon Bogaty <[email protected]> wrote: > > Brilliant Baptiste, thank you. I've setup proper logging and a longer > > timeout: > > > > global > > user nobody > > group nobody > > daemon > > nbproc 4 > > maxconn 204800 > > log /dev/log local0 info > > log /dev/log local0 notice > > > > tune.bufsize 16384 # 16k > > tune.rcvbuf.server 141312 # 128k > > > > defaults > > log global > > option httplog > > option dontlognull > > > > mode http > > backlog 32768 > > maxconn 204800 > > > > timeout connect 120ms # how long to try to connect to a > > backend > > timeout queue 120ms # how long a request can wait for > a > > backend before 503ing > > timeout server 15s # how long to wait for response from > > backend before 503ing > > > > timeout client 60000ms # how long to wait for data from > > clients (exchanges) > > timeout http-keep-alive 60000ms # how long to keep keepalive > sessions > > when inactive > > > > option abortonclose > > no option forceclose > > option http-no-delay > > option nolinger > > > > I'm down to 1 or 2 504s in a 1000... It's weird though, doesn't seem to > be > > making a difference whether I go to 10s or 15, still got these last one > or > > two pesky 504s. Anything else I could be missing? > > > > > > On Wed, May 28, 2014 at 10:04 AM, Baptiste <[email protected]> wrote: > >> > >> On Wed, May 28, 2014 at 3:56 PM, Jon Bogaty <[email protected]> wrote: > >> > Hi Baptiste, > >> > I'm sorry, I should clarify, I meant 504. It's really quite prevalent, > >> > at > >> > least 4/10 at times, sometimes 8/10... > >> > > >> > I'm using: > >> > HA-Proxy version 1.4.24 2013/06/17 > >> > > >> > This is more or less the way the entirety of the configuration is: > >> > global > >> > user nobody > >> > group nobody > >> > daemon > >> > nbproc 4 > >> > maxconn 204800 > >> > > >> > tune.bufsize 16384 # 16k > >> > tune.rcvbuf.server 141312 # 128k > >> > > >> > defaults > >> > log global > >> > option tcplog > >> > option dontlognull > >> > > >> > mode http > >> > backlog 32768 > >> > maxconn 204800 > >> > > >> > timeout connect 120ms # how long to try to connect > to a > >> > backend > >> > timeout queue 120ms # how long a request can wait > for > >> > a > >> > backend before 503ing > >> > timeout server 120ms # how long to wait for response > >> > from > >> > backend before 503ing > >> > > >> > timeout client 60000ms # how long to wait for data from > >> > clients (exchanges) > >> > timeout http-keep-alive 60000ms # how long to keep keepalive > >> > sessions > >> > when inactive > >> > > >> > option abortonclose > >> > no option forceclose > >> > option http-no-delay > >> > option nolinger > >> > > >> > frontend openx > >> > bind *:9010 > >> > default_backend bidder9010 > >> > > >> > backend bidder9010 > >> > balance roundrobin > >> > server bid001 10.1.1.50:9010 weight 1 maxconn 51200 check > >> > server bid002 10.1.1.112:9010 weight 1 maxconn 51200 check > >> > server bid003 10.1.1.113:9010 weight 1 maxconn 51200 check > >> > server bid004 10.1.1.114:9010 weight 1 maxconn 51200 check > >> > server bid005 10.1.1.115:9010 weight 1 maxconn 51200 check > >> > server bid007 10.1.1.117:9010 weight 1 maxconn 51200 check > >> > server bid008 10.1.1.118:9010 weight 1 maxconn 51200 check > >> > server bid009 10.1.1.119:9010 weight 1 maxconn 51200 check > >> > server bid010 10.1.1.120:9010 weight 1 maxconn 51200 check > >> > server bid011 10.1.1.127:9010 weight 1 maxconn 51200 check > >> > server bid012 10.1.1.128:9010 weight 1 maxconn 51200 check > >> > server bid013 10.1.1.126:9010 weight 1 maxconn 51200 check > >> > server bid014 10.1.1.203:9010 weight 1 maxconn 51200 check > >> > server bid015 10.1.1.204:9010 weight 1 maxconn 51200 check > >> > server bid016 10.1.1.205:9010 weight 1 maxconn 51200 check > >> > > >> > Basically haproxy balances a set of those bidder backends from port > 9010 > >> > to > >> > 9080... Does that clarify things? > >> > > >> > > >> > On Wed, May 28, 2014 at 9:40 AM, Baptiste <[email protected]> wrote: > >> >> > >> >> On Wed, May 28, 2014 at 3:31 PM, Jon Bogaty <[email protected]> > wrote: > >> >> > Hi, > >> >> > I have two questions... I am having a lot of problems with 500 > errors > >> >> > from > >> >> > haproxy and I am wondering if these could be two culprits: > >> >> > > >> >> > Is there an equivalent method for disabling Nagle Algorithm in TCP > >> >> > mode? > >> >> > I've looked everywhere and it seems that TCP NO DELAY is not a flag > >> >> > within > >> >> > haproxy. Only http mode seems to include the option. > >> >> > > >> >> > Could nbproc possibly have a negative effect as opposed to a > >> >> > beneficial > >> >> > one? > >> >> > Is it possible that by setting nbproc to four we're actually > creating > >> >> > problems with scalability and with the number of concurrent working > >> >> > connections? > >> >> > > >> >> > I can post pieces of my haproxy.cfg if it helps explain how I'm > >> >> > building > >> >> > out > >> >> > the load balancing. I feel like somewhere in my config there's > >> >> > something > >> >> > incorrectly tuned that's causing connection problems. Any help > would > >> >> > be > >> >> > greatly appreciated. > >> >> > > >> >> > Thanks! > >> >> > Jon > >> >> > >> >> > >> >> Hi Jon, > >> >> > >> >> Please post at least your HAProxy version, how you built/installed > it, > >> >> etc... > >> >> configuration, logs showing the errors are welcome too. > >> >> > >> >> Note that HAProxy is not supposed to generate any 500 errors (only > >> >> 502, 503, 504) > >> >> > >> >> Baptiste > >> > > >> > > >> > >> > >> Could you please turn on "option httplog" and remove the tcplog option? > >> > >> 504 means the server did not answer fast enough (longuer than the > >> timeout server). > >> Just increase the timeout server a bit and see what happens. > >> We usually set it up to a few seconds (less than 20). > >> > >> Baptiste > > > > > > if the page takes longer than 15s to be generated, then the issue must > be in the application. > now, the log can tell you which URLs has generated the 504 ;) > It's up to you to play. > > Baptiste >
