Reserving memory is critical especially if you over commit. Haproxy degrades extremely quickly when things swap. Don't allow swapping inside or outside of the vm. Reserving a small amount of CPU is good. That said I doubt if either of these is your problem. It's probably more related to the setup under Ubuntu.
It works great with Centos 5 under ESXi. (Some with native kernel, and some with custom kernel so I can run in transparent mode). How many connections/sec are you handling? I could give my tuning info for Centos. My vms are handling a little over 1000 connections/sec without any issue, but know idea if there might be a cap of what is reasonable under a vm, but it should be at least 10x that. Do some basic tests, like try the following from your haproxy vm to your server: ping -A -s 18000 -c 1000 -q 10.3.4.84 PING 10.3.4.84 (10.3.4.84) 18000(18028) bytes of data. --- 10.3.4.84 ping statistics --- 1000 packets transmitted, 1000 received, 0% packet loss, time 1303ms rtt min/avg/max/mdev = 0.669/1.132/10.712/0.675 ms, ipg/ewma 1.304/1.768 ms The 18k packet size is good as it tests full length packets which is a much better test then default 64 bytes. It's surprising how often something will mess up on full length packets due to problems in their vlan taggin assignments, etc... (The above is testing from a vm on one physical server to a vm on a different physical server). So make sure basic stuff like ping works. If that's ok, I suggest tryin Centos instead of ubuntu. ----- Original Message ----- From: "Baptiste" <[email protected]> To: "Christophe Rahier" <[email protected]> Cc: "Willy Tarreau" <[email protected]>, "Tim Korves" <[email protected]>, [email protected] Sent: Wednesday, September 14, 2011 6:01:45 AM Subject: Re: Establishing connection lasts long first you should reserve resources on your ESX server for your HAProxy box, to ensure it will never wait for the hypervisor to find some resources for it. Then, in the VM, you should tune networking sysctl (tcp port range, tcp rmem, wmem, the syn backlog, etc...) you can also do CPU affinity per process to ensure no processes will bother HAProxy (usefull under huge load). last chance, move out from the ESX to a physical server :) good luck On Wed, Sep 14, 2011 at 10:04 AM, Christophe Rahier <[email protected]> wrote: > Hi, > > In my case, I searched on my Debian server but I don't find nf_conntrack > ... > > Concerning tuning my whole system, why not but in which direction? :-) > > Regards > > Christophe > > > > Le 14/09/11 00:34, « Willy Tarreau » <[email protected]> a écrit : > >>On Tue, Sep 13, 2011 at 03:31:21PM +0200, Tim Korves wrote: >>> Hi there, >>> >>> >This has nothing to see with haproxy but more how your hypervisor >>> >manages VMs which doesn't do anything :) >>> >>> thanks for the information. Do you have any tips regarding vmWare ESXi >>> 4.1 ? >> >>First, the common suspect : are all of your VMs on the same HV ? Quite >>commonly, what is observed is that involving multiple VMs adds huge >>delays, especially if more total logical CPUs are allocated than the >>total physical ones (which is to be expected because the HV has to >>schedule everyone in turn). >> >>The other common issue when you see something slowing down through >>haproxy is a lack of tuning of the OS, specifically conntrack, but >>this is not related to the VMs at all, it's general. Just in case, >>unload the nf_conntrack module from the machine running haproxy. >>If it changes anything, then either you have to definitely remove >>it because you don't use it, or you need to tune your whole system, >>which is often quite an amusing exercise in VMs ;-) >> >>Regards, >>Willy >> >> >> > > > >
