Hi Babtiste, yes we do have change that:
[root@srvA ~]# cat /etc/sysconfig/network-scripts/route-eth0 default via 192.168.1.254 dev eth0 table 700 [root@srvA ~]# cat /etc/sysconfig/network-scripts/rule-eth0 from 192.168.1.1 lookup 700 The setup is working just fine, for a while. When we try to overwhelm the proxy with thousands of requests from single IP (within same network or outside, doesnt matter) it gets stuck after a while. Then again turning off tproxy settings (source 0.0.0.0 usesrc clientip) solves the problem and all starts working again. V Mon, 26 Jan 2015 13:57:24 +0100 Baptiste <bed...@gmail.com> napsáno: > Hi Zbynek, > > Have you changed the default gateway of your server? > traffic from server to client must pass through HAProxy box. > > In your case, I guess HAProxy sends a SYN to the server and the > servers sends the S/A to the client directly, bypassing HAProxy. > > Baptiste > > > On Mon, Jan 26, 2015 at 1:24 PM, Zbyněk Rozman <zbynek.roz...@stable.cz> > wrote: > > Hi, > > > > we are currently facing problem with connection clogging with this > > configuration. > > The client after a while (random number of requests ~few hundreds) reaches > > "timeout connect" settings on backend server - proxy returns 503 Service > > Unavailable No server is available to handle this request. Other clients on > > different IPs are connecting to server just fine. > > Strace on haproxy when its not working shows just: Resource temporarily > > unavailable. > > After a while it starts working again for random number of requests and > > then 503 occurs again. > > When switched to another backend server it starts working again. Switching > > back to broken backend - still request timeouts for affected clients. > > It looks like it is only affecting clients behind public IP address with > > high traffic. > > We have tried to change various sysctl settings, with no success so far. > > When we remove the haproxy setting "source 0.0.0.0 usesrc clientip" all > > works just fine without clogging - so it looks like its tproxy problem, but > > we are unable to find solution. > > > > *** backend server uname -a: > > Linux backend 2.6.32-504.3.3.el6.x86_64 #1 SMP Wed Dec 17 01:55:02 UTC 2014 > > x86_64 x86_64 x86_64 GNU/Linux > > > > *** haproxy -vv > > HA-Proxy version 1.5.8 2014/10/31 > > Copyright 2000-2014 Willy Tarreau <w...@1wt.eu> > > > > Build options : > > TARGET = linux26 > > CPU = generic > > CC = gcc > > CFLAGS = -O2 -g -fno-strict-aliasing > > OPTIONS = USE_LINUX_TPROXY=1 USE_REGPARM=1 USE_OPENSSL=1 USE_PCRE=1 > > > > Default settings : > > maxconn = 2000, bufsize = 16384, maxrewrite = 8192, maxpollevents = 200 > > > > Encrypted password support via crypt(3): yes > > Built without zlib support (USE_ZLIB not set) > > Compression algorithms supported : identity > > Built with OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013 > > Running on OpenSSL version : OpenSSL 1.0.1e-fips 11 Feb 2013 > > OpenSSL library supports TLS extensions : yes > > OpenSSL library supports SNI : yes > > OpenSSL library supports prefer-server-ciphers : yes > > Built with PCRE version : 7.8 2008-09-05 > > PCRE library supports JIT : no (USE_PCRE_JIT not set) > > Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT > > IP_FREEBIND > > > > Available polling systems : > > epoll : pref=300, test result OK > > poll : pref=200, test result OK > > select : pref=150, test result OK > > Total: 3 (3 usable), will use epoll. > > > > *** haproxy server uname -a > > Linux haproxy-test 2.6.32-504.3.3.el6.x86_64 #1 SMP Wed Dec 17 01:55:02 UTC > > 2014 x86_64 x86_64 x86_64 GNU/Linux > > > > *** haproxy server settings: > > > > iptables -t mangle -N DIVERT > > iptables -t mangle -A PREROUTING -p tcp --sport 80 -m socket -j DIVERT > > iptables -t mangle -A PREROUTING -p tcp --sport 81 -m socket -j DIVERT > > iptables -t mangle -A PREROUTING -p tcp --sport 82 -m socket -j DIVERT > > iptables -t mangle -A PREROUTING -p tcp --sport 443 -m socket -j DIVERT > > iptables -t mangle -A DIVERT -j MARK --set-mark 1 > > iptables -t mangle -A DIVERT -j ACCEPT > > > > ip rule add fwmark 1 lookup 100 > > ip route add local 0.0.0.0/0 dev lo table 100 > > > > sysctl -w net.ipv4.ip_forward=1 > > sysctl -w net.ipv4.ip_nonlocal_bind=1 > > > > *** haproxy.cfg: > > > > frontend front > > bind IP:80 > > mode http > > > > default_backend WWW > > capture cookie SERVERID len 32 > > capture request header Content-Length len 10 > > reqadd X-Forwarded-Proto:\ http > > timeout connect 6s > > > > backend WWW > > server srvA 192.168.1.1:80 weight 1 check source 0.0.0.0 > > usesrc clientip > > > > > > *** sysctl.cfg: > > > > kernel.shmmax = 4294967295 > > kernel.shmall = 268435456 > > net.ipv4.ip_nonlocal_bind = 1 > > net.ipv4.ip_forward = 1 > > net.ipv4.tcp_slow_start_after_idle=0 > > net.nf_conntrack_max = 200000 > > net.netfilter.nf_conntrack_tcp_timeout_established = 1800 > > net.core.rmem_max=16777216 > > net.core.wmem_max=16777216 > > net.ipv4.tcp_rmem=4096 87380 16777216 > > net.ipv4.tcp_wmem=4096 65536 16777216 > > net.core.netdev_max_backlog = 30000 > > net.ipv4.tcp_max_syn_backlog = 8192 > > net.ipv4.ip_local_port_range = 1024 65535 > > net.core.somaxconn = 40000 > > net.ipv4.tcp_max_tw_buckets = 1440000 > > net.ipv4.tcp_synack_retries = 2 > > net.ipv4.tcp_syn_retries = 2 > > > > -- > > S pozdravem > > > > Zbyněk Rozman > > Linux System Administrator > > > > zbynek.roz...@stable.cz > > 480 020 191 > > > > Stable.cz s.r.o. | www.stable.cz > > Skelná 29, 466 02, Jablonec nad Nisou > > Tel.: 480 020 191, fax: 480 020 091 > > > > napoveda.stable.cz > > > > www.stable.cz | www.otoman.cz | www.kvapem.cz | www.multak.cz > >
