i did some further debugging and also enabled the rsyslog on my ubuntu server but i am unable to find the root cause of this issue???
here is what my admin stat looks like (if you see the swebcluster/load1-app has all values 0) webcluster Queue Session rate Sessions Bytes Denied Errors Warnings Server Cur Max Limit Cur Max Limit Cur Max Limit Total LbTot In Out Req Resp Req Conn Resp Retr Redis Status LastChk Wght Act Bck Chk Dwn Dwntme Thrtle Frontend 1 32 - 1 6 2000 203 85538 1850676 0 0 0 OPEN load1-app 0 0 - 0 2 0 2 - 109 109 30470 1198316 0 0 0 0 0 2h30m UP L7OK/200 in 1ms 1 Y - 0 0 0s - load2-app 0 0 - 0 32 0 6 - 72 12 37869 395381 0 0 0 0 0 2h30m UP L7OK/200 in 73ms 1 Y - 1 0 0s - Backend 0 0 0 32 0 6 2000 181 121 85538 1850676 0 0 0 0 0 0 2h30m UP 2 2 0 0 0s swebcluster Queue Session rate Sessions Bytes Denied Errors Warnings Server Cur Max Limit Cur Max Limit Cur Max Limit Total LbTot In Out Req Resp Req Conn Resp Retr Redis Status LastChk Wght Act Bck Chk Dwn Dwntme Thrtle Frontend 0 5 - 0 6 2000 28 61312 267759 0 0 0 OPEN load1-app 0 0 - 0 0 0 0 - 0 0 0 0 0 0 0 0 0 2h30m UP L7OK/200 in 1ms 2 Y - 0 0 0s - load2-app 0 0 - 0 5 0 6 - 28 28 61312 267759 0 0 0 0 0 2h30m UP L7OK/200 in 75ms 2 Y - 0 0 0s - Backend 0 0 0 5 0 6 2000 28 28 61312 267759 0 0 0 0 0 0 2h30m UP 4 2 0 0 0s the output from my /var/log/haproxy.log is Mar 2 20:03:04 localhost haproxy[21654]: 105:59775 [02/Mar/2011:20:03:04.402] webcluster webcluster/load2-app 1/15/82 5214 -- 6/5/5/5/0 0/0 Mar 2 20:03:04 localhost haproxy[21654]: 105:59774 [02/Mar/2011:20:03:04.401] webcluster webcluster/load2-app 0/17/85 808 -- 5/4/4/4/0 0/0 Mar 2 20:03:04 localhost haproxy[21654]: 105:59773 [02/Mar/2011:20:03:04.401] webcluster webcluster/load2-app 0/17/87 804 -- 4/3/3/3/0 0/0 Mar 2 20:03:04 localhost haproxy[21654]: 105:59776 [02/Mar/2011:20:03:04.403] webcluster webcluster/load2-app 3/12/85 1432 -- 3/2/2/2/0 0/0 Mar 2 20:03:04 localhost haproxy[21654]: 105:59771 [02/Mar/2011:20:03:04.397] webcluster webcluster/load2-app 1/21/92 4874 -- 2/1/1/1/0 0/0 Mar 2 20:03:04 localhost haproxy[21654]: 105:59772 [02/Mar/2011:20:03:04.397] webcluster webcluster/load2-app 4/17/204 20075 -- 1/0/0/0/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59778 [02/Mar/2011:20:03:05.132] webcluster webcluster/load2-app 3/2/68 556 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59780 [02/Mar/2011:20:03:05.138] webcluster webcluster/load2-app 4/36/104 556 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59779 [02/Mar/2011:20:03:05.135] webcluster webcluster/load2-app 2/41/107 1325 -- 5/4/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59781 [02/Mar/2011:20:03:05.138] webcluster webcluster/load2-app 9/31/107 1188 -- 4/3/3/3/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59782 [02/Mar/2011:20:03:05.142] webcluster webcluster/load2-app 3/33/103 4392 -- 3/2/2/2/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59777 [02/Mar/2011:20:03:05.132] webcluster webcluster/load2-app 0/5/145 13095 -- 2/1/1/1/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59784 [02/Mar/2011:20:03:05.225] webcluster webcluster/load2-app 1/27/92 482 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59788 [02/Mar/2011:20:03:05.287] webcluster webcluster/load2-app 2/9/76 529 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59787 [02/Mar/2011:20:03:05.286] webcluster webcluster/load2-app 3/9/77 481 -- 5/4/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59786 [02/Mar/2011:20:03:05.285] webcluster webcluster/load2-app 3/10/79 481 -- 4/3/3/3/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59785 [02/Mar/2011:20:03:05.278] webcluster webcluster/load2-app 6/13/85 482 -- 3/2/2/2/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59792 [02/Mar/2011:20:03:05.312] webcluster webcluster/load2-app 1/25/91 532 -- 3/2/1/1/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59793 [02/Mar/2011:20:03:05.343] webcluster webcluster/load2-app 0/35/91 323 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59797 [02/Mar/2011:20:03:05.407] webcluster webcluster/load2-app 0/8/76 391 -- 6/5/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59794 [02/Mar/2011:20:03:05.404] webcluster webcluster/load2-app 0/11/79 323 -- 5/4/3/3/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59795 [02/Mar/2011:20:03:05.404] webcluster webcluster/load2-app 2/9/80 1115 -- 4/3/3/3/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59798 [02/Mar/2011:20:03:05.428] webcluster webcluster/load2-app 1/29/127 9343 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59796 [02/Mar/2011:20:03:05.407] webcluster webcluster/load2-app 0/8/176 30906 -- 5/4/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59802 [02/Mar/2011:20:03:05.515] webcluster webcluster/load2-app 0/22/90 1110 -- 5/4/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59800 [02/Mar/2011:20:03:05.512] webcluster webcluster/load2-app 0/25/139 10066 -- 4/3/3/3/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59799 [02/Mar/2011:20:03:05.481] webcluster webcluster/load2-app 2/12/181 26587 -- 3/2/2/2/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59803 [02/Mar/2011:20:03:05.585] webcluster webcluster/load2-app 3/31/104 532 -- 5/4/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59801 [02/Mar/2011:20:03:05.512] webcluster webcluster/load2-app 3/22/239 50544 -- 6/5/5/5/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59805 [02/Mar/2011:20:03:05.681] webcluster webcluster/load2-app 0/20/78 533 -- 5/4/4/4/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59806 [02/Mar/2011:20:03:05.684] webcluster webcluster/load2-app 1/16/75 514 -- 4/3/3/3/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59804 [02/Mar/2011:20:03:05.669] webcluster webcluster/load2-app 1/30/90 384 -- 3/2/2/2/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59808 [02/Mar/2011:20:03:05.712] webcluster webcluster/load2-app 0/26/142 447 -- 3/2/1/1/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59807 [02/Mar/2011:20:03:05.711] webcluster webcluster/load2-app 0/27/143 514 -- 2/1/0/0/0 0/0 Mar 2 20:03:05 localhost haproxy[21654]: 105:59809 [02/Mar/2011:20:03:05.781] webcluster webcluster/load2-app 74/3/140 448 -- 1/0/0/0/0 0/0 Mar 2 20:03:06 localhost haproxy[21654]: 105:59812 [02/Mar/2011:20:03:05.940] webcluster webcluster/load2-app 0/38/102 477 -- 3/2/2/2/0 0/0 Mar 2 20:03:06 localhost haproxy[21654]: 105:59811 [02/Mar/2011:20:03:05.938] webcluster webcluster/load2-app 2/38/106 449 -- 2/1/1/1/0 0/0 Mar 2 20:03:06 localhost haproxy[21654]: 105:59810 [02/Mar/2011:20:03:05.936] webcluster webcluster/load2-app 1/41/144 6066 -- 1/0/0/0/0 0/0 Mar 2 20:03:06 localhost haproxy[21654]: 105:59813 [02/Mar/2011:20:03:06.109] webcluster webcluster/load2-app 2/25/89 1056 -- 1/0/0/0/0 0/0 Mar 2 20:03:19 localhost haproxy[21654]: 11:41581 [02/Mar/2011:20:02:59.467] swebcluster swebcluster/load2-app 0/32/19713 4366 -- 0/0/0/0/0 0/0 Mar 2 20:04:15 localhost haproxy[21654]: 124:50153 [02/Mar/2011:20:04:15.298] webcluster webcluster/load1-app 35/0/200 30906 -- 1/0/0/0/0 0/0 Mar 2 20:04:15 localhost haproxy[21654]: 124:50154 [02/Mar/2011:20:04:15.777] webcluster webcluster/load1-app 40/1/156 26587 -- 1/0/0/0/0 0/0 Mar 2 20:04:17 localhost haproxy[21654]: 124:50157 [02/Mar/2011:20:04:17.219] webcluster webcluster/load1-app 74/1/235 50544 -- 1/0/0/0/0 0/0 Mar 2 20:04:17 localhost haproxy[21654]: 105:49229 [02/Mar/2011:20:04:17.877] webcluster webcluster/load2-app 0/20/84 808 -- 6/5/5/5/0 0/0 Mar 2 20:04:17 localhost haproxy[21654]: 105:49231 [02/Mar/2011:20:04:17.878] webcluster webcluster/load2-app 2/18/90 804 -- 5/4/4/4/0 0/0 Mar 2 20:04:17 localhost haproxy[21654]: 105:49227 [02/Mar/2011:20:04:17.870] webcluster webcluster/load2-app 4/23/107 5214 -- 4/3/3/3/0 0/0 Mar 2 20:04:17 localhost haproxy[21654]: 105:49232 [02/Mar/2011:20:04:17.878] webcluster webcluster/load2-app 2/18/104 1432 -- 3/2/2/2/0 0/0 Mar 2 20:04:17 localhost haproxy[21654]: 105:49230 [02/Mar/2011:20:04:17.878] webcluster webcluster/load2-app 1/19/106 4874 -- 2/1/1/1/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49228 [02/Mar/2011:20:04:17.876] webcluster webcluster/load2-app 1/21/168 20075 -- 1/0/0/0/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49236 [02/Mar/2011:20:04:18.217] webcluster webcluster/load2-app 1/40/111 556 -- 6/5/5/5/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49241 [02/Mar/2011:20:04:18.234] webcluster webcluster/load2-app 1/22/108 481 -- 5/4/4/4/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49239 [02/Mar/2011:20:04:18.234] webcluster webcluster/load2-app 0/24/109 1325 -- 4/3/3/3/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49240 [02/Mar/2011:20:04:18.234] webcluster webcluster/load2-app 1/23/111 482 -- 3/2/2/2/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49237 [02/Mar/2011:20:04:18.230] webcluster webcluster/load2-app 1/27/115 4392 -- 2/1/1/1/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49238 [02/Mar/2011:20:04:18.231] webcluster webcluster/load2-app 2/24/163 13095 -- 6/5/5/5/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49242 [02/Mar/2011:20:04:18.348] webcluster webcluster/load2-app 1/29/51 532 -- 5/4/4/4/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49243 [02/Mar/2011:20:04:18.363] webcluster webcluster/load2-app 3/12/82 323 -- 6/5/5/5/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49246 [02/Mar/2011:20:04:18.367] webcluster webcluster/load2-app 1/10/80 391 -- 5/4/4/4/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49244 [02/Mar/2011:20:04:18.367] webcluster webcluster/load2-app 1/10/83 1115 -- 4/3/3/3/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49251 [02/Mar/2011:20:04:18.493] webcluster webcluster/load2-app 3/2/51 1110 -- 6/5/5/5/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49247 [02/Mar/2011:20:04:18.428] webcluster webcluster/load2-app 0/30/126 9343 -- 5/4/4/4/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49245 [02/Mar/2011:20:04:18.367] webcluster webcluster/load2-app 1/10/198 30906 -- 4/3/3/3/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49249 [02/Mar/2011:20:04:18.466] webcluster webcluster/load2-app 2/29/103 10066 -- 4/3/2/2/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49248 [02/Mar/2011:20:04:18.429] webcluster webcluster/load2-app 1/28/200 26587 -- 5/4/3/3/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49250 [02/Mar/2011:20:04:18.470] webcluster webcluster/load2-app 5/22/208 50544 -- 5/4/3/3/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49254 [02/Mar/2011:20:04:18.598] webcluster webcluster/load2-app 2/18/92 514 -- 4/3/2/2/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49253 [02/Mar/2011:20:04:18.578] webcluster webcluster/load2-app 1/38/117 384 -- 3/2/1/1/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49256 [02/Mar/2011:20:04:18.650] webcluster webcluster/load2-app 3/5/65 448 -- 3/2/1/1/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49257 [02/Mar/2011:20:04:18.699] webcluster webcluster/load2-app 1/37/146 6066 -- 2/1/0/0/0 0/0 Mar 2 20:04:18 localhost haproxy[21654]: 105:49252 [02/Mar/2011:20:04:18.569] webcluster webcluster/load2-app 306/22/389 533 -- 1/0/0/0/0 0/0 Mar 2 20:04:21 localhost haproxy[21654]: 105:49255 [02/Mar/2011:20:04:21.590] webcluster webcluster/load2-app 1/21/89 477 -- 1/0/0/0/0 0/0 Mar 2 20:04:21 localhost haproxy[21654]: 105:49258 [02/Mar/2011:20:04:21.713] webcluster webcluster/load2-app 14/11/88 1056 -- 1/0/0/0/0 0/0 Mar 2 20:04:32 localhost haproxy[21654]: 11:41766 [02/Mar/2011:20:04:11.209] swebcluster swebcluster/load2-app 0/4/21579 4366 -- 0/0/0/0/0 0/0 Mar 2 20:04:59 localhost haproxy[21654]: 140:45978 [02/Mar/2011:20:04:59.439] webcluster webcluster/load1-app 0/0/137 210 -- 0/0/0/0/0 0/0 Mar 2 20:04:59 localhost haproxy[21654]: 140:45980 [02/Mar/2011:20:04:59.758] webcluster webcluster/load1-app 41/0/66 26587 -- 0/0/0/0/0 0/0 Mar 2 20:05:40 localhost haproxy[21654]: 105:59828 [02/Mar/2011:20:05:40.241] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0 Mar 2 20:05:42 localhost haproxy[21654]: 105:59829 [02/Mar/2011:20:05:42.642] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0 Mar 2 20:05:44 localhost haproxy[21654]: 105:59830 [02/Mar/2011:20:05:44.969] webcluster webcluster/<NOSRV> -1/-1/1 12338 PR 0/0/0/0/0 0/0 Mar 2 20:05:47 localhost haproxy[21654]: 105:59831 [02/Mar/2011:20:05:47.551] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0 Mar 2 20:05:48 localhost haproxy[21654]: 105:59832 [02/Mar/2011:20:05:48.751] webcluster webcluster/<NOSRV> -1/-1/0 12338 PR 0/0/0/0/0 0/0 Mar 2 20:05:55 localhost haproxy[21654]: 105:59833 [02/Mar/2011:20:05:55.856] webcluster webcluster/<NOSRV> -1/-1/1 12338 PR 0/0/0/0/0 0/0 --- On Wed, 3/2/11, Amol <[email protected]> wrote: From: Amol <[email protected]> Subject: Re: SSL Load balancing question To: [email protected] Date: Wednesday, March 2, 2011, 3:52 PM I have a question regarding this config of haproxy, i see that all my connections are only going to app1 and app2 has no connections at all, i changed it last night from balance roundrobin to balance source, do i have to flush any files, etc. Why would this be occuring? listen swebcluster 1.1.1.8:8000 mode tcp balance source option ssl-hello-chk option httpchk HEAD /check.txt HTTP/1.0 server app1 159:80 maxconn 1024 check server app2 148:80 maxconn 1024 check --- On Sat, 2/26/11, Amol <[email protected]> wrote: From: Amol <[email protected]> Subject: SSL Load balancing question To: [email protected] Date: Saturday, February 26, 2011, 4:06 PM So i have a haproxy 1.4 setup with the following configuration global log 127.0.0.1 local0 log 127.0.0.1 local1 notice maxconn 20000 ulimit-n 65536 user haproxy group haproxy daemon stats socket /var/run/haproxy/haproxy.sock mode 0600 level admin defaults log global option tcplog option dontlognull retries 3 option redispatch contimeout 5000 clitimeout 50000 srvtimeout 200000 listen swebcluster 1.1.42.18:800 mode tcp balance roundrobin option ssl-hello-chk option httpchk HEAD /check.txt HTTP/1.0 server p-server-app1 1.1.4.59:80 maxconn 5000 check server p-server-app2 1.1.7.148:80 maxconn 5000 check one thing i have noticed is that when i run a long running request on the load balanced url the whole setup just hangs, like its waiting on this one request to complete, (yes my application does frequent request and since it is roundrobin may be it is going to the server that is already handling one request? my question is if there is a better configuration to handle this, so my app can use the same server and other users are not affected by this 1 request? the cookie setup is not possible in tcp correct?
