> > With "forever" you mean longer then 1m ?
Yes, unfortunately forever means forever, not just 1 minute. I have already tried to wait several minutes (e.g. more than 10 min) and the number of backend connections reported by Datadog remains the same (e.g. ~200). When I restart HAProxy then the number of connections drops to the normal value (0-2 connections). > retry-on all-retryable-errors I can't do that. I prefer to abort failed requests and return an error to the client, instead of retrying them for the following reason: You have to make sure the application has a replay protection mechanism built in such as a unique transaction IDs passed in requests, or that replaying the same request has no consequence, or it is very dangerous to use any retry-on value beside "conn-failure" and "none". On Tue, Oct 1, 2019 at 12:02 PM Aleksandar Lazic <[email protected]> wrote: > Am 01.10.19 um 11:18 schrieb Marco Colli: > > Here's my configuration: > > > > $ haproxy -vv > > HA-Proxy version 2.0.7-1ppa1~bionic 2019/09/28 - https://haproxy.org/ > > [snipp] > > > $ cat /etc/haproxy/haproxy.cfg > > global > > log /dev/loglocal0 > > log /dev/loglocal1 notice > > chroot /var/lib/haproxy > > stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd > listeners > > stats timeout 30s > > user haproxy > > group haproxy > > daemon > > > > maxconn 16384 > > > > nbproc 1 > > nbthread 4 > > cpu-map auto:1/1-4 0-3 > > > > # Default SSL material locations > > ca-base /etc/ssl/certs > > crt-base /etc/ssl/private > > > > # See: > > > https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate > > ssl-default-bind-ciphers ... > > ssl-default-bind-ciphersuites ... > > ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 > no-tls-tickets > > tune.ssl.default-dh-param 2048 > > > > defaults > > logglobal > > modehttp > > option httpchk HEAD /health HTTP/1.1\r\nHost:\ example.com > > \r\nX-Forwarded-Proto:\ https > > optionhttplog > > optiondontlognull > > option dontlog-normal > > option forwardfor > > option http-server-close > > option redispatch > > timeout client 10s > > timeout client-fin 5s > > timeout http-request 5s > > timeout server 30s > > timeout server-fin 10s > > timeout connect 10s > > timeout queue 10s > > errorfile 400 /etc/haproxy/errors/400.http > > errorfile 403 /etc/haproxy/errors/403.http > > errorfile 408 /etc/haproxy/errors/408.http > > errorfile 500 /etc/haproxy/errors/500.http > > errorfile 502 /etc/haproxy/errors/502.http > > errorfile 503 /etc/haproxy/errors/503.http > > errorfile 504 /etc/haproxy/errors/504.http > > > > listen stats > > bind :8000 > > bind-process 1 > > mode http > > stats enable > > stats hide-version > > stats realm HAProxy\ Stats > > stats uri / > > stats auth theuser:thepassword > > > > frontend www-frontend > > bind :::80 v4v6 > > bind :::443 v4v6 ssl crt /etc/ssl/private/ev-2019.pem > > default_backend www-backend > > compression algo gzip > > compression type text/html text/css text/javascript > > application/javascript application/json > > > > backend www-backend > > http-request redirect prefix https://%[hdr(host),regsub(^www\.,,i)] if { > > hdr_beg(host) -i www. } > > http-request add-header X-Forwarded-Proto https > > redirect scheme https if !{ ssl_fc } > > balance roundrobin > > default-server maxconn 256 inter 10s fall 3 rise 2 check > > server web0 10.113.220.155:6000 <http://10.113.220.155:6000> > > server web1 10.113.221.156:6000 <http://10.113.221.156:6000> > > server web2 10.113.222.157:6000 <http://10.113.222.157:6000> > > > > > > On Tue, Oct 1, 2019 at 11:02 AM Aleksandar Lazic <[email protected] > > <mailto:[email protected]>> wrote: > > > > Hi. > > > > Am 01.10.19 um 10:46 schrieb Marco Colli: > > > Hello! > > > > > > I use HAProxy to load balance HTTP(S) traffic to some web servers. > Web servers > > > then connect to a database. I have noticed that when we restart > the database > > > some errors occur (and that is normal during the restart). > > > > > > However the problem is that **a few hundreds connections remain > open from > > > HAProxy to the Puma web servers forever**. That slow down HAProxy. > > With "forever" you mean longer then 1m ? > > I would try to add `retry-on all-retryable-errors` in the default section > and > see if the behavour changes. > http://cbonte.github.io/haproxy-dconv/2.0/configuration.html#4.2-retry-on > > > > When we restart HAProxy then everything works fine again and the > number of > > > backend connections drops to zero, which is the normal value > since we use > > option > > > http-server-close. We have also configured the following timeouts > but nothing > > > has changed (some connections to backend remain open forever): > > > > > > timeout client 10s > > > timeout client-fin 5s > > > timeout http-request 5s > > > timeout server 30s > > > timeout server-fin 10s > > > timeout connect 10s > > > timeout queue 10s > > > > > > HAProxy Version: 2.0 > > > > Please can you post the full haproxy -vv as there are many fixes in > the laster > > versions. > > > > Are there any checks in the config? > > Can you share the (minimal) config so that we can see some more > information's > > about your setup. > > > > Regards > > Aleks > > > >
