Hi Lukas, On Sun, Mar 17, 2013 at 09:02:37PM +0100, Lukas Tribus wrote: > > Hi, > > please CC the list, so everyone is seeing your responses. > > > > Ubuntu 12.04 LTS, Kernel 3.5, Haproxy 1.4.22 from repo > > please provide the output from "haproxy -vv" nonetheless. It > should have been compiled with the linux2628 target, as this > will enable features like tcp splicing. > > > > No, I need to handle 10GE, maybe even more using 1 loadbalancer > > Hardware is: 2 x E5620 XEON, 96GB RAM, Dual 10Gbit/s Intel NIC. > > This kind of performance can be accomplished (see [1]), but you need to > configure the kernel and haproxy accordingly. > > Also please make sure you read the documentation very carefully > about every setting you make. > > You disabled for example splicing, epoll and sepoll, which I > think doesn't make any sense and will slow down haproxy significantly. > > > > Each download is limited to 700Kbit/s and files are from 40MB till 900MB. > > So you are not hosting small html, css, js files but only big files for one > time download, is this correct? > > Then TCP splicing will be very effective and you will not see much CPU > cycles spend in haproxy, but in the kernel. This also allows you to > use just a single haproxy process. > > > > > option http-no-delay > > >Same here, does your application really needs this? > > Yes, backend is also supporting it > > You said the files you're serving are at least 40MB; for sure this is > not a realtime application and http-no-delay means higher CPU usage > and more overhead due to more packets on the network, so the proxy will > be slower with this configuration. > > > > > Why are you disabling maxaccept, can you elaborate this? > > Was thinking that 80k limit of connections will not be enough > > I'm not sure I understand what you mean; I suggest you remove this > command as well, it may harm performance of established connections. > > > So what I'm suggesting is: > > - remove the following configuration: > > noepoll > > nokqueue > > nopoll > > nosepoll > > nosplice > > - also remove: > > option http-no-delay > > tune.maxaccept > > - configure TCP splicing according to the doc [2] > - remove "nbproc 20", a single process will be enough > - pin haproxy to a specific core, and the interrupt handling to > another core, but on the same CPU, so that shared CPU cache > can be used between interrupt handling and haproxy > - then monitor CPU usage under heavy load
I just wanted to confirm that I 100% agree with you on all the tuning parameters, that's what I'm used to do too, and I reached 20 Gbps a few weeks ago with 10MB files this way. Best regards, Willy
