Hi John, On Thu, May 22, 2014 at 08:28:55AM +0200, John-Paul Bader wrote: > Hey, > > over the last 10 days we tested how much traffic one server with > haproxy-1.5dev25 could take and yesterday we've managed to saturate one > machine to the point where the load was equal to the number of real > cores in the machine and the listening queues began to fill up.
OK. > We're running on a 12 core Intel Xeon E5-2620 v2 2.10GHz with 32GB of > RAM and with two Intel i350 dual port network cards. One for public IP, > the other one for the LAN. > > At the point when the listening queues filled up we had about 76000 open > sockets, a connection rate of 4800/s and about 68Mbit/s of traffic. > > System and interrupt load was quite low all the time and the haproxy > processes were at about 80% CPU > > Our traffic is half HTTPS, half HTTP - coming from all over the world > with all kinds of latencies and with rather short lived but burst like > sessions. > > I guess there are still optimizations possible and we'll try to find > them but so far we're happy with the results. > > Maybe someone here can comment whether they think these are good number > or if we should be able to push much more through one machine. When SSL is involved, it's very hard to say whether it's normal or not, because the CPU load can vary ten-fold depending on the rekeying rate. For example, if you're running with few visitors doing a lot of requests, there are rarely new key renegotiation so the handshake are cheap. However if you have many visitors doing request in a while, the renegotiation rate will be much higher. I don't know if we can easily know whether a handshake involved a key renegotiation, in which case we could maintain separate stats for these. Given you're talking about 80% CPU for haproxy, I'm assuming you're talking about user CPU then, which would indicate it's mostly rekeying that is happening. For more normal connections, you can try to play with the various ciphers, tls versions (eg: 1.1 is more expensive than 1.2), and maybe even try to disable tickets to see if anything changes. I don't remember the exact numbers we got on our appliances, but I'm having vague memories of something like 3000 keys/s/core on a core i7 3.3 GHz for 1024 bit keys, so if we consider we have 1000 keys/s per gigahertz, it should be about 25k/s on your system. If you're running in 2048 bits, then you can divide performance by about 4.5 and that would give about what you're observing. Maybe users with high traffic SSL production servers can provide some input here. Best regards, Willy
