Hi Ari,
If the backend servers are responding slowly for some reason, all
haproxy nodes will crash.
i configured queue to 1 and set up reasonable server and client
timeouts on haproxy
however if for some reason connections builds up on haproxy crashes
and the machine becomes unresponsive (all haproxy nodes)
I don't see any kernel errors.
1. how can i debug this ?
2. what is the best practice to protect haproxy from such situation ?
So there is nothing in the logs, nothing in dmesg? No OOM events, no
crashes?
Then how do you know haproxy is crashing, if there is not a single log
line about it? The node becomes completely unresponsive, like not even
ping and ssh works? Then it is likely not a haproxy problem (but do
provide "top" output anyway). Sounds more like a stateful firewall in
front of haproxy with exhausted resources.
Also, how do you know connections are building up on haproxy? Can you
see that on the stat interface? If yes, how does that look like exactly?
Please share the haproxy configuration and check your stateful firewalls
for resource exhaustion.
Regards,
Lukas