On Tue, Aug 08, 2017 at 07:16:17AM +0200, Claudio Kuenzler wrote:
> I've set "hard limits" with maxconn for each backend server but it seems
> that established (keep-alive) connections are not accounted for in the
> stats. This leads to HAProxy allowing more connections to the backend
> server than actually defined with the maxconn value.
> To me it looks like only connections with current data-transfer are
> accounted for in the CUR statistics, idle connections somehow drop out and
> HAProxy then allows additional connections to the backend, causing it to
> stale (it cannot handle more than 7 sessions). Am I wrong? Did I
> misinterpret the maxconn purpose?
No you're totally right and it's by design. The thing is, you don't want
to leave requests waiting in a server's queue while the server has a ton
of idle connections. The vast majority of servers nowadays have a
dispatching frontend which is mostly insensitive to idle connections, and
only really see outstanding requests. This has been even more true since
all browsers started to implement the pre-connect feature a few years ago,
establishing idle connections to sites you've recently visited just in
case you'd want to visit them again, resulting in a huge amount of idle
connections on servers. So when using server-side keep-alive we continue
to ensure that the server never has to process more than a given number
of outstanding requests, and idle connections are not accounted for.
Now the question is, does it cause any problem for you or is it just that
it came as a surprize and you were worried that it could cause problems ?
The possible alternative would be to have an option to say that idle
connections are accounted for and that some of them will be killed before
passing a new connection to the server, but that will significantly reduce
the efficiency of server-side keep-alive.
If you're really short on server-side connections and want to optimize
them as much as possible, you can try to enable "http-reuse". It will
allow sharing of idle connections between frontend connections so that
a request may be sent over an existing connection. It is the way to
achieve the lowest number of concurrent connections on the server side.
But not all applications support this (most do nowadays), you need to
check (eg: some try to retrieve the source address once per connection
for logging purposes for example).