That's why the ML is there :) Cheers
On Wed, Oct 2, 2013 at 9:49 AM, Tom Huybrechts <[email protected]> wrote: > I can't believe I missed that... > Thanks! > > > On Wed, Oct 2, 2013 at 7:03 AM, Baptiste <[email protected]> wrote: >> >> Hi Tom, >> >> Your global maxconn parameter is not responsible of your issue. >> That said it looks very low, so I recommend you to increase it (1000 >> sounds good to start, depends on your traffic level) >> >> Your server maxconn will queue connection when a server reaches 32 >> connections only. >> Is there any reason why you're having so low value here as well? >> It means the 33th connection (and the next ones as well) would be >> queued on the server for 5s max (timeout queue). >> Increasing the timeout queue won't fix the issue, it will only >> increase the latency on the client side. >> I would recommend increasing server's maxconn unless you have good >> reasons to have so low values. If so, then you'll have to add more >> servers in your backend to increase backend capacity. >> >> I also recommend reading HAProxy's documentation related to maxconn. >> >> Baptiste >> >> >> On Tue, Oct 1, 2013 at 3:28 PM, Tom Huybrechts <[email protected]> >> wrote: >> > >> > I have maxconn 256 and timeout 5s. >> > >> > Full configuration below. I don't think there's anything special about >> > it... >> > >> > Tom >> > >> > >> > >> > >> > >> > global >> > daemon >> > maxconn 256 >> > log localhost local0 >> > >> > defaults >> > balance source >> > log global >> > option log-separate-errors >> > option tcplog >> > mode tcp >> > timeout connect 5000ms >> > timeout client 360000ms >> > timeout server 1800000ms >> > >> > listen stats :9876 >> > mode http >> > stats enable >> > stats uri / >> > >> > listen dicom_104 >> > bind 10.233.20.140:104 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 104 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 104 inter >> > 2000 >> > listen dicom_105 >> > bind 10.233.20.140:105 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 105 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 105 inter >> > 2000 >> > listen dicom_106 >> > bind 10.233.20.140:106 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 106 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 106 inter >> > 2000 >> > listen dicom_107 >> > bind 10.233.20.140:107 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 107 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 107 inter >> > 2000 >> > listen dicom_108 >> > bind 10.233.20.140:108 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 108 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 108 inter >> > 2000 >> > listen dicom_109 >> > bind 10.233.20.140:109 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 109 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 109 inter >> > 2000 >> > >> > listen hl7_2310 >> > bind 10.233.20.140:2310 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2310 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2310 inter >> > 2000 >> > listen hl7_2311 >> > bind 10.233.20.140:2311 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2311 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2311 inter >> > 2000 >> > listen hl7_2350 >> > bind 10.233.20.140:2350 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2350 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2350 inter >> > 2000 >> > listen hl7_2351 >> > bind 10.233.20.140:2351 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2351 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2351 inter >> > 2000 >> > listen hl7_2352 >> > bind 10.233.20.140:2352 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2352 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2352 inter >> > 2000 >> > listen hl7_2353 >> > bind 10.233.20.140:2353 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2353 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2353 inter >> > 2000 >> > listen hl7_2354 >> > bind 10.233.20.140:2354 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2354 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2354 inter >> > 2000 >> > listen hl7_2355 >> > bind 10.233.20.140:2355 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2355 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2355 inter >> > 2000 >> > listen hl7_2356 >> > bind 10.233.20.140:2356 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2356 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2356 inter >> > 2000 >> > listen hl7_2357 >> > bind 10.233.20.140:2357 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2357 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2357 inter >> > 2000 >> > listen hl7_2358 >> > bind 10.233.20.140:2358 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2358 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2358 inter >> > 2000 >> > listen hl7_2359 >> > bind 10.233.20.140:2359 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2359 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2359 inter >> > 2000 >> > listen hl7_2360 >> > bind 10.233.20.140:2360 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2360 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2360 inter >> > 2000 >> > listen hl7_2361 >> > bind 10.233.20.140:2361 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 2361 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 2361 inter >> > 2000 >> > >> > listen collaborator_7070 >> > bind 10.233.20.140:7070 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter >> > 2000 >> > listen collaborator_5222 >> > bind 10.233.20.140:5222 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter >> > 2000 >> > listen collaborator_5223 >> > bind 10.233.20.140:5223 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter >> > 2000 >> > listen collaborator_7443 >> > bind 10.233.20.140:7443 >> > balance roundrobin >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 7070 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 7070 inter >> > 2000 >> > >> > frontend binary >> > bind >> > >> > 10.233.20.140:4447,10.233.20.140:7900,10.233.20.140:9080,10.233.20.140:2762 >> > default_backend binary >> > >> > frontend https >> > bind 10.233.20.140:443,10.233.20.140:8443 >> > default_backend https >> > >> > frontend http >> > bind 10.233.20.140:80,10.233.20.140:8080 >> > default_backend http >> > >> > backend binary >> > balance roundrobin >> > option httpchk GET /status >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 80 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 80 inter >> > 2000 >> > >> > backend https >> > balance roundrobin >> > option httpchk GET /status >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 80 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 80 inter >> > 2000 >> > >> > backend http >> > mode http >> > balance roundrobin >> > option httpchk GET /status >> > appsession JSESSIONID len 52 timeout 3h >> > server 10.233.20.130 10.233.20.130 maxconn 32 check port 80 inter >> > 2000 >> > server 10.233.23.123 10.233.23.123 maxconn 32 check port 80 inter >> > 2000 >> > >> > >> > >> > On Tue, Oct 1, 2013 at 2:23 PM, Baptiste <[email protected]> wrote: >> >> >> >> Hi tom, >> >> >> >> Can you share us your configuration? >> >> HAProxy seems to be queueing the connection (maxconn set to 1??) and >> >> the timeout queue (setup to 5s in your case??) has expired so the >> >> connection is not forwarded to the server. >> >> >> >> Of course, I want you to send us some comments with your >> >> configuration, we need to understand why you did configure your >> >> settings :) >> >> >> >> Baptiste >> >> >> >> >> >> >> >> On Tue, Oct 1, 2013 at 2:18 PM, Tom Huybrechts >> >> <[email protected]> >> >> wrote: >> >> > Hi all, >> >> > >> >> > I'm using haproxy 1.4.23 on Windows Server 2008 to proxy tcp >> >> > connections >> >> > to >> >> > a single backend server. Usually this is going well, but occasionally >> >> > a >> >> > client has issues: connections get dropped and they are unable to >> >> > make >> >> > new >> >> > ones. >> >> > >> >> > Investigation of the logs show server-side timeouts: >> >> > >> >> > [01/Oct/2013:12:06:10.653] binary binary/<NOSRV> 5015/-1/5015 0 sQ >> >> > 40/33/33/0/0 0/1 >> >> > [01/Oct/2013:12:06:10.653] binary binary/<NOSRV> 5015/-1/5015 0 sQ >> >> > 39/32/32/0/0 0/2 >> >> > [01/Oct/2013:12:06:15.934] binary binary/<NOSRV> 5000/-1/5000 0 sQ >> >> > 42/36/36/0/0 0/1 >> >> > [01/Oct/2013:12:06:17.231] binary binary/<NOSRV> 5062/-1/5062 0 sQ >> >> > 42/36/36/0/0 0/2 >> >> > [01/Oct/2013:12:06:17.247] binary binary/<NOSRV> 5046/-1/5046 0 sQ >> >> > 41/35/35/0/0 0/3 >> >> > [01/Oct/2013:12:06:19.856] binary binary/<NOSRV> 5015/-1/5015 0 sQ >> >> > 40/34/34/0/0 0/4 >> >> > [01/Oct/2013:12:06:19.856] binary binary/<NOSRV> 5015/-1/5015 0 sQ >> >> > 39/33/33/0/0 0/5 >> >> > [01/Oct/2013:12:06:21.012] binary binary/<NOSRV> 5031/-1/5031 0 sQ >> >> > 38/32/32/0/0 0/5 >> >> > >> >> > I have no idea why: the server is not under heavy load. >> >> > >> >> > I have a wireshark dump of one such occasion. I expected to at least >> >> > see >> >> > SYN >> >> > packets going from haproxy to server in this timeframe, but there >> >> > where >> >> > none. In the meanwhile other connections from haproxy to the same >> >> > server >> >> > continued without problems. >> >> > >> >> > Any hints to what's wrong, or what I can do to investigate further? >> >> > >> >> > Tom >> >> > >> > >> > > >
