Hello, I'm keep digging on this problem and that's what I've found so far: I have another one frontend/backend pair handled by the same instance, and it looks like whenever this backend goes slow, everything goes slow. Why one slow backend affects others? Any recommendations on how can I avoid this?
Thanks a lot, Dmitry. 2016-10-18 15:53 GMT+03:00 Dmitry Maslov <[email protected]>: > Hello, > > I'm experience latency spikes during the bussiness hours which affects all > services behind single HAProxy instance (both tcp and http). > > Here's example lines from log: > > Oct 18 14:29:34 localhost haproxy[173339]: 192.168.4.130:53568 > <http://192.168.24.130:53568> [18/Oct/2016:14:29:31.248] server_tcp > b_server_tcp/tcps0 654/633/3371 1049 -- 72186/700/700/162/0 0/0 > Oct 18 14:31:56 localhost haproxy[173339]: 192.168.4.130:53630 > <http://192.168.24.130:53630> [18/Oct/2016:14:31:54.715] server_tcp > b_server_tcp/tcps3 352/444/2268 1049 -- 73099/701/701/128/0 0/0 > Oct 18 14:31:59 localhost haproxy[173339]: 192.168.4.130:53632 > <http://192.168.24.130:53632> [18/Oct/2016:14:31:57.053] server_tcp > b_server_tcp/tcps4 337/456/2431 1049 -- 72798/702/701/143/0 0/0 > Oct 18 14:32:01 localhost haproxy[173339]: 192.168.4.130:53634 > <http://192.168.24.130:53634> [18/Oct/2016:14:31:59.461] server_tcp > b_server_tcp/tcps0 291/314/2405 1049 -- 72294/701/701/163/0 0/0 > > If i do the same request directly to backend (they are on the same machine > with HAProxy), I'm getting response in 150-200 ms. > > As you can see from logs, Tw and Tc is very high despite that backend and > server queues are empty. Spikes starts whenever actconn is going over 50k. > > I'm using HAProxy 1.5.11, here is a part of my haproxy.conf (related to > log lines above): > > frontend server_tcp > bind :80 > bind :443 > mode tcp > log global > option tcplog > maxconn 5000 > timeout client 150s > option clitcpka > default_backend b_server_tcp > > backend b_server_tcp > mode tcp > option tcplog > log global > option srvtcpka > timeout server 1m > balance roundrobin > server tcps0 192.168.2.200:9090 <http://192.168.20.200:9090> > maxconn 1000 send-proxy-v2 > server tcps1 192.168.2.200:9091 <http://192.168.20.200:9091> > maxconn 1000 send-proxy-v2 > server tcps2 192.168.2.200:9092 <http://192.168.20.200:9092> > maxconn 1000 send-proxy-v2 > server tcps3 192.168.2.200:9093 <http://192.168.20.200:9093> > maxconn 1000 send-proxy-v2 > server tcps4 192.168.2.200:9094 <http://192.168.20.200:9094> > maxconn 1000 send-proxy-v2 > > Any suggestion on how to investigate this problem? > > Tanks a lot, > Dmitry. >
