I can't believe I missed that... Thanks!
On Wed, Oct 2, 2013 at 7:03 AM, Baptiste <[email protected]> wrote: > Hi Tom, > > Your global maxconn parameter is not responsible of your issue. > That said it looks very low, so I recommend you to increase it (1000 > sounds good to start, depends on your traffic level) > > Your server maxconn will queue connection when a server reaches 32 > connections only. > Is there any reason why you're having so low value here as well? > It means the 33th connection (and the next ones as well) would be > queued on the server for 5s max (timeout queue). > Increasing the timeout queue won't fix the issue, it will only > increase the latency on the client side. > I would recommend increasing server's maxconn unless you have good > reasons to have so low values. If so, then you'll have to add more > servers in your backend to increase backend capacity. > > I also recommend reading HAProxy's documentation related to maxconn. > > Baptiste > > > On Tue, Oct 1, 2013 at 3:28 PM, Tom Huybrechts <[email protected]> > wrote: > > > > I have maxconn 256 and timeout 5s. > > > > Full configuration below. I don't think there's anything special about > it... > > > > Tom > > > > > > > > > > > > global > > daemon > > maxconn 256 > > log localhost local0 > > > > defaults > > balance source > > log global > > option log-separate-errors > > option tcplog > > mode tcp > > timeout connect 5000ms > > timeout client 360000ms > > timeout server 1800000ms > > > > listen stats :9876 > > mode http > > stats enable > > stats uri / > > > > listen dicom_104 > > bind 10.233.20.140:104 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 104 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 104 inter > 2000 > > listen dicom_105 > > bind 10.233.20.140:105 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 105 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 105 inter > 2000 > > listen dicom_106 > > bind 10.233.20.140:106 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 106 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 106 inter > 2000 > > listen dicom_107 > > bind 10.233.20.140:107 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 107 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 107 inter > 2000 > > listen dicom_108 > > bind 10.233.20.140:108 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 108 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 108 inter > 2000 > > listen dicom_109 > > bind 10.233.20.140:109 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 109 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 109 inter > 2000 > > > > listen hl7_2310 > > bind 10.233.20.140:2310 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2310 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2310 inter > 2000 > > listen hl7_2311 > > bind 10.233.20.140:2311 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2311 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2311 inter > 2000 > > listen hl7_2350 > > bind 10.233.20.140:2350 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2350 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2350 inter > 2000 > > listen hl7_2351 > > bind 10.233.20.140:2351 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2351 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2351 inter > 2000 > > listen hl7_2352 > > bind 10.233.20.140:2352 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2352 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2352 inter > 2000 > > listen hl7_2353 > > bind 10.233.20.140:2353 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2353 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2353 inter > 2000 > > listen hl7_2354 > > bind 10.233.20.140:2354 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2354 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2354 inter > 2000 > > listen hl7_2355 > > bind 10.233.20.140:2355 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2355 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2355 inter > 2000 > > listen hl7_2356 > > bind 10.233.20.140:2356 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2356 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2356 inter > 2000 > > listen hl7_2357 > > bind 10.233.20.140:2357 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2357 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2357 inter > 2000 > > listen hl7_2358 > > bind 10.233.20.140:2358 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2358 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2358 inter > 2000 > > listen hl7_2359 > > bind 10.233.20.140:2359 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2359 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2359 inter > 2000 > > listen hl7_2360 > > bind 10.233.20.140:2360 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2360 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2360 inter > 2000 > > listen hl7_2361 > > bind 10.233.20.140:2361 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2361 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2361 inter > 2000 > > > > listen collaborator_7070 > > bind 10.233.20.140:7070 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter > 2000 > > listen collaborator_5222 > > bind 10.233.20.140:5222 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter > 2000 > > listen collaborator_5223 > > bind 10.233.20.140:5223 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter > 2000 > > listen collaborator_7443 > > bind 10.233.20.140:7443 > > balance roundrobin > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter > 2000 > > > > frontend binary > > bind > > 10.233.20.140:4447,10.233.20.140:7900,10.233.20.140:9080, > 10.233.20.140:2762 > > default_backend binary > > > > frontend https > > bind 10.233.20.140:443,10.233.20.140:8443 > > default_backend https > > > > frontend http > > bind 10.233.20.140:80,10.233.20.140:8080 > > default_backend http > > > > backend binary > > balance roundrobin > > option httpchk GET /status > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 80 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 80 inter > 2000 > > > > backend https > > balance roundrobin > > option httpchk GET /status > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 80 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 80 inter > 2000 > > > > backend http > > mode http > > balance roundrobin > > option httpchk GET /status > > appsession JSESSIONID len 52 timeout 3h > > server 10.233.20.130 10.233.20.130 maxconn 32 check port 80 inter > 2000 > > server 10.233.23.123 10.233.23.123 maxconn 32 check port 80 inter > 2000 > > > > > > > > On Tue, Oct 1, 2013 at 2:23 PM, Baptiste <[email protected]> wrote: > >> > >> Hi tom, > >> > >> Can you share us your configuration? > >> HAProxy seems to be queueing the connection (maxconn set to 1??) and > >> the timeout queue (setup to 5s in your case??) has expired so the > >> connection is not forwarded to the server. > >> > >> Of course, I want you to send us some comments with your > >> configuration, we need to understand why you did configure your > >> settings :) > >> > >> Baptiste > >> > >> > >> > >> On Tue, Oct 1, 2013 at 2:18 PM, Tom Huybrechts < > [email protected]> > >> wrote: > >> > Hi all, > >> > > >> > I'm using haproxy 1.4.23 on Windows Server 2008 to proxy tcp > connections > >> > to > >> > a single backend server. Usually this is going well, but occasionally > a > >> > client has issues: connections get dropped and they are unable to make > >> > new > >> > ones. > >> > > >> > Investigation of the logs show server-side timeouts: > >> > > >> > [01/Oct/2013:12:06:10.653] binary binary/<NOSRV> 5015/-1/5015 0 sQ > >> > 40/33/33/0/0 0/1 > >> > [01/Oct/2013:12:06:10.653] binary binary/<NOSRV> 5015/-1/5015 0 sQ > >> > 39/32/32/0/0 0/2 > >> > [01/Oct/2013:12:06:15.934] binary binary/<NOSRV> 5000/-1/5000 0 sQ > >> > 42/36/36/0/0 0/1 > >> > [01/Oct/2013:12:06:17.231] binary binary/<NOSRV> 5062/-1/5062 0 sQ > >> > 42/36/36/0/0 0/2 > >> > [01/Oct/2013:12:06:17.247] binary binary/<NOSRV> 5046/-1/5046 0 sQ > >> > 41/35/35/0/0 0/3 > >> > [01/Oct/2013:12:06:19.856] binary binary/<NOSRV> 5015/-1/5015 0 sQ > >> > 40/34/34/0/0 0/4 > >> > [01/Oct/2013:12:06:19.856] binary binary/<NOSRV> 5015/-1/5015 0 sQ > >> > 39/33/33/0/0 0/5 > >> > [01/Oct/2013:12:06:21.012] binary binary/<NOSRV> 5031/-1/5031 0 sQ > >> > 38/32/32/0/0 0/5 > >> > > >> > I have no idea why: the server is not under heavy load. > >> > > >> > I have a wireshark dump of one such occasion. I expected to at least > see > >> > SYN > >> > packets going from haproxy to server in this timeframe, but there > where > >> > none. In the meanwhile other connections from haproxy to the same > server > >> > continued without problems. > >> > > >> > Any hints to what's wrong, or what I can do to investigate further? > >> > > >> > Tom > >> > > > > > >
