Our server (a recent 8-core xeon server) is able to handle a few
rounds of benchmarks, but then connections just seem to start hanging
somewhere -- they don't show up in haproxy_stats. At the same time,
every server listed on the stats page starts going red (even though
they are clearly fine -- you can hit them directly without problem).
It does not seem to be memory-related (no bulging swap file at any
rate).
My config file is as follows:
________
global
log 127.0.0.1 local0 notice
#log 127.0.0.1 local1 alert
daemon
nbproc 2
maxconn 32000
ulimit-n 65536
# and uid, gid, daemon, maxconn...
defaults
# setup logging
log global
option tcplog
option httplog
option dontlognull
# setup statistics pages
stats enable
stats uri /haproxy_stats
stats refresh 10
stats auth xxxxxxxxxx:xxxxxxxxxxx
mode http
retries 3
option abortonclose
option redispatch
timeout connect 80s # time in queue
timeout client 30s # data from client (browser)
timeout server 80s # data from server (mongrel)
frontend rails *:7000
# set connection limits
maxconn 700
default_backend mongrels
backend mongrels
fullconn 80
option httpchk
option abortonclose
contimeout 4000
balance roundrobin
server webapp01-101 10.2.0.3:8101 minconn 1 maxconn 5 check
inter 2s rise 1 fall 2
server webapp01-102 10.2.0.3:8102 minconn 1 maxconn 5 check
inter 2s rise 1 fall 2
<snip> lots more of those over several machines... </snip>
## this is the service I'm testing against
frontend webservers *:81
mode tcp
default_backend nginx
backend nginx
mode tcp
balance leastconn
contimeout 1000
server webapp01-http 10.2.0.3:8080 check
server webapp02-http 10.2.0.4:8080 check
server webapp04-http 10.2.0.17:8080 check
server webapp05-http 10.2.0.18:8080 check
___________
the test I'm running is:
ab -n 10000 -c 20 "nowhere.com:81/images/layout/blank.gif"
... from another machine on the LAN. It runs fine (and pretty fast)
several times in a row, but then something happens; it'll hang
somewhere in the test, and none of the requests will show up on the
haproxy stats page. Sometimes Apache (which feeds into haproxy for its
balancer for Rails requests) starts showing 503s as well.
I'm wondering if I have some bad combination of haproxy configuration,
or if I should look elsewhere for the cause... the system is running
FC7.
cheers
