Hi Jiri,
Am 04.01.2017 um 11:38 schrieb Jiri Mencak:
Hi,
we are using HAProxy with its default 2000 maxconn limit and a listen
block:
listen stats :1936
mode http
monitor-uri /healthz
which we use to check HAProxy's "health" by external HTTP probes. The
behaviour I'm seeing is that once the default 2000 maxconn limit is
reached,
HAProxy stops listening
[...]
As far as I can see, staying with the HTTP probe model, we can
increase the
global maxconn limit or/and increase the health-check's timeout period.
Do you see any other options?
Increasing global maxconn is the correct thing to do.
Here's why:
They are multiple maxconn "levels" that work independently from each other.
We have maxconn at global (process) level, at listen/frontend level, and
server level.
What you wanna do is make sure that the queuing happens in the lower
tiers first:
1. maxconn at server level is exhausted, haproxy will queue or
redispatch to another
backend server
2. maxconn at bind/frontend/listen level is exhausted, haproxy will stop
accepting
new connections and the kernel will queue for this particular level
(e.g. the specific frontend)
3. maxconn at the process level is exhausted, the kernel will queue for
all frontends
on this kernel of the specific process
The reason is impact. If one frontend/listen section is overloaded and
we are hitting its "own"
maxconn, other sections remain unaffected (your http probes in the
listen section for example).
But if the global maxconn is exhausted, everything is gonna be affected
(which is exactly what
you are seeing).
What are the dangers of setting the global maxconn limits really high
apart
from increased HAProxy's memory usage?
Its really only memory (haproxy and kernel). But that also means that
when you
go beyond what your box can handle in ram and swap, you gonna get haproxy
OOM-killed.
If you have 1GB of memory to spare on this box, you can set global maxconn
to 20000 without any problems (estimate from the docs). And because each
listen/frontend section is still bound to the per section maxconn
default (2000),
your monitor section won't be affected by exhaustion in the other sections.
Consider tuning per listen/frontend maxconn as well.
maxconn is the single most import performance setting in haproxy, and it
should
be adjusted for each use case, considering available memory.
Also see:
https://cbonte.github.io/haproxy-dconv/1.6/configuration.html#4.2-maxconn
Regards,
Lukas
