Hi Lukas, On Tue, May 19, 2015 at 2:06 PM, Lukas Tribus <[email protected]> wrote:
> Hi Pawel, > > Hi. > > > > We've tried migrating haproxy from 1.4.22 to 1.5.2. As a result we > > experienced a serious performance impact. The only thing that was > > changed was hapxory version. We tried 1.5.12 afterwards, but that > > didn't really help. This is running on AWS Amazon Linux 64bit. 1.5.2 > > was AWS stock, 1.5.12 we built ourselves. > > Check conntrack settings on both proxy and the backends. Different haproxy > behavior may trigger conntrack limits by using more connections. > I believe we don't have conntrack in any form, on either the backends or haproxy machines. I see you set "option httpclose" on the default. Was this set in both > 1.4.22 and 1.5.x instances? Yes. > This settings should theoretically make > haproxy behave exactly the same. > So think that somehow, 1.5 was creating or keeping a lot more open connections at a time, and depriving the kernel, or its own limits of available connections? > In its absence, the behavior would indeed have changed, from the old > default: > option http-tunnel [1] > > to: > option http-keep-alive [2] > > > One thing I can suggest is to replace "option httpclose" in the default > section with the following configuration: > option http-keep-alive > option prefer-last-server > timeout http-keep-alive 5s (or whatever keep alive timeout works for you) > I personally don't see any reason for having any connections left kept alive. We want to minimize the amount of open files, we've had problems with keep-alive choking us quite quickly. I see the doc says that httpclose is "...is deprecated since what it does is very cheap but not reliable", which I guess means that it won't work with all the servers, but our servers honor the Connection:close header quite fine, so I don't see a reason to change that. I guess what I should do - is try 1.5 during "quiet time", and compare the environment (open fds, etc) with 1.4, and see what is different... > Regards, > > Lukas > > > [1] > http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option%20http-tunnel > [2] > http://cbonte.github.io/haproxy-dconv/configuration-1.5.html#option%20http-keep-alive > -- With best of best regards Pawel S. Veselov
