On Wed, Jul 23, 2014 at 11:37 AM, Tamim Khan <[email protected]> wrote:
> Hi everyone, > > I am having some issues with upgrading from HAProxy 1.4.18 to 1.5.2 using > the same config. With 1.5.2, I am seeing issues where HAProxy is becoming > unresponsive a few minutes after I have started it up. When this issue > starts to occur the stats page starts to load very slowly (or not at all) > and backend servers start to be marked as down even though external > monitoring is showing that they are perfectly fine. > > After comparing the stats page for 1.4.18 vs 1.5.2, I have noticed that > with 1.5.2 the session rate on my backend is very high compared to 1.4.18 > (1000+ vs 60) while the current amount of sessions is very low for 1.5.2 > and much higher for 1.4.18 (3 vs 1000+). This makes me think that for some > reason HAProxy is not respecting http keep alive and is re-establishing > every single connection. Another reason behind my thinking, is that HAProxy > 1.5.2 seems to work correctly when all references to "mode http" are > changed to "mode tcp". > > I have verified both my backend servers (apache) and the servers infront > of HAProxy have the appropriate keep alive headers and are using HTTP 1.1. > The sanitized config I am using can be viewed here: > http://pastebin.com/2gL4xKu7. > > Does anyone have any ideas as to why this is occurring? > Pastebin won't let me see your config (under heavy load) right now. However, I'd guess that you might be seeing an impact from the http default changing from tunnel mode to keep-alive mode and that this higher request rate to your backends has overwhelmed the conntrack tables on your proxy. Check your system logs for reports of dropped packets. You could also be seeing source port exhaustion on the proxy but that doesn't usually generate errors in system logs -- just frequent failure to make connections after a timeout. See "option http-tunnel" in the docs. If tunnel mode works for you it can be explicitly enabled in 1.5 to get the old default behavior. It might also be possible to use keep-alive to your backends but that is a new feature in 1.5 and I've not tried that out at all yet. -Bryan
