Hi Dmitriy, On Tue, Jul 26, 2011 at 03:51:20AM +0400, Dmitriy Samsonov wrote: > Hi! > > First of all thanks too all on this list for helping me to setup and > configure haproxy. Thanks to Willy for haproxy. I thought it is a good idea > to save somebody like me some time configuring Dell's servers for haproxy to > meet DDoS or other traffic-intensive task. Solution is - "get rid of bnx2 > 1Gbit NIC".
Hehe :-) > There is an option in Del''s servers to install Intel's 10Gbit > NIC - it is working a way faster (x3, x5) then broadcom. Intel's 10G NICs are fast, but generally hard to tune, you have a hard trade-off between data rate and latency. Basically you tune them for large or small packets but not both. Still for DDoS they might be well suited. > But anyway it is > impossible to have one server to filter 10Gbit DDoS attack, at least it is > very far from default configuration, so I failed to find solution. 10G DDoS is something very hard to resist ! > But as haproxy is really good software to handle DDoS you can use amazon's > cloud. And that was my final solution. Amazon's servers are taking 400Mbit > per node, so installing 60 EC2 nodes I've managed to filter DDoS traffic at > 24Gbit/s (It was somewhere around 10*10^6 requests per second, or ~150-160k > session rate per node) without any problems. Yes, there was some balancing > issues, and some failures for couple of minutes, but it's nothing comparing > to typical DDoS expirience. Wow! I'm impressed! But did you perform the tests from within Amazon's cloud or from the outside ? I don't know what their external peering is, and I'm wondering how you managed to send 24 Gbps over the internet. Also, what's the cost of running 60 of these nodes ? > And question to Willy: What hardware in your opinion is the best to run > haproxy on to serve lot's of HTTP requests 99% percent of which are trash to > be blocked? My best results were achieved on Core i5 3.3 GHz (dual core, 4 threads). In short, I bind network IRQs to core zero and haproxy to core one and the two remaining threads ensure there is some rope left for system management, SSH, etc. With this you can run at 100% CPU all the day if you want. I managed to get haproxy to dynamically filter 300k connections per second on such a configuration. That's SYN, SYN/ACK, ACK, RST, based on a source IP's connection rate. Depending on the type of DDoS you're dealing with, it may be worth distributing the NIC's IRQs to multiple cores so that the kernel's work can scale (eg: emit a SYN/ACK). It might even be worth having multiple haproxy processes, because when you're blocking a DDoS, you don't really mind about monitoring, stats, health checks, etc... You only want your machine to be as fast as possible, and doing so is possible with multi-queues. You then need to spread your NICs IRQs to all cores and bind as many haproxy processes as cores (and maually pin them). The CPU-NIC affinity will not be good on the backend but that's not the issue since you're dealing with a frontend where you want to reject most of your traffic. Best regards, Willy