Hello,
I have set up haproxy on two virtual (Xen) machines, listening to two
virtual IP addresses (heartbeat).
It's loadbalancing nginx, varnish, memcache, php-fpm and mysql and
functionally it works. I seem to be running into some sort of resource
limitation however; When I run siege on the website it always ends up
saying:
siege aborted due to excessive socket failure; you
can change the failure threshold in $HOME/.siegerc
Transactions: 1038 hits
Availability: 50.34 %
Elapsed time: 5.26 secs
Data transferred: 8.24 MB
Response time: 0.07 secs
Transaction rate: 197.34 trans/sec
Throughput: 1.57 MB/sec
Concurrency: 14.05
Successful transactions: 1038
Failed transactions: 1024
Longest transaction: 1.04
Shortest transaction: 0.01
The number of failed transactions and the number of transactions is
always the same.
I've tweaked the kernel with some things I've found on the net:
net.core.rmem_max=16777216
net.core.wmem_max=16777216
net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.ipv4.tcp_fin_timeout = 3
net.ipv4.tcp_tw_recycle = 0
net.core.netdev_max_backlog = 30000
net.ipv4.tcp_no_metrics_save=1
net.core.somaxconn = 262144
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 262144
net.ipv4.tcp_synack_retries = 2
net.ipv4.ip_nonlocal_bind=1
net.ipv4.tcp_syn_retries = 2
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 1024 65023
net.ipv4.tcp_max_tw_buckets = 400000
I've included my config as well.
Any ideas as to what I have forgotten about that makes the system run
out of resources here? I can't find anything in the syslog that
indicates any problems. It might be the Xen host logs anything but have
not checked that yet, or it may also be the BSD firewall (in front of
these two haproxy servers) that runs out of resources but haven't
checked those yet as I'd first like to get my haproxy setup checked.
Thanks,
Fred Leeflang
global
#log 127.0.0.1 local0
log 127.0.0.1 local1 notice
#log loghost local0 info
maxconn 8192
# ulimit-n 16413
#chroot /usr/share/haproxy
user haproxy
group haproxy
daemon
# debug
# quiet
# nbproc 4
noepoll
nopoll
defaults
log global
option dontlognull
retries 3
option redispatch
contimeout 5000
clitimeout 50000
srvtimeout 50000
listen stats lb1:8081,lb2:8081
mode http
stats enable
stats hide-version
stats realm HaProxy\ Statistics
stats uri /
stats auth xxx:xxxxx
frontend memcache
bind lb1:11211,lb2:11211
default_backend memcache
frontend cacti
bind lb1:8080,lb2:8080
mode http
default_backend cacti_http
frontend php
bind lb1:9000,lb2:9000
default_backend php
frontend clubsites
bind lb1:80,lb2:80
mode http
option httpclose
option forwardfor
option httplog
log global
# log the name of the virtual server
capture request header Host len 64
acl clubsites_nocache hdr_end(host) -i domain.nl
acl clubsites_images path_end .gif .png .jpg
acl clubsites_jscss path_end .css .js
# Images get sent to a different varnish backend than jscss
use_backend varnish if clubsites_images or clubsites_jscss
#use_backend varnish_jscss if clubsites_jscss
use_backend nginx if clubsites_nocache
default_backend nginx
frontend mysql_read
bind lb1:3306,lb2:3306
default_backend mysql_read
backend memcache
mode tcp
server memcache1 clubsite155:11211 check
server memcache-bkup clubsite165:11211 check backup
backend nginx
mode http
option httpchk
server http1 clubsite150:80 check
server http-bkup clubsite160:80 check backup
backend php
server php151 clubsite151:9000 check
server php152 clubsite152:9000 check
server php153 clubsite153:9000 check
server lb1 lb1:9000 check backup
backend varnish
mode http
option httpchk
server varnish clubsite15:80 check
server varnish_bkup clubsite16:80 check backup
backend cacti_http
mode http
option httpchk
server cacti clubsite165:80 check
backend mysql_read
mode tcp
option httpchk
server mysql clubsite154:3306 check port 9201
server mysql-bkup clubsite164:3306 check port 9201 backup
