Are you running irqbalance? It may help distribute network interface interrupts

Sent from my iPhone

On 18 Jul 2011, at 03:42, Dmitriy Samsonov <dmitriy.samso...@gmail.com> wrote:

> My test setup is three Dell r410 servers (dual Intel(R) Xeon(R) CPU X5650  @ 
> 2.67GHz - 24 threads total, 128Gb RAM) all connected to 1Gbps network.
> 
> One server is haproxy, configured to block all requests with 
> 'Accept-Encoding: none':
> 
> global
>       daemon
>       maxconn 80000
>       option forwardfor
>       retries 10
> 
> frontend public
>         bind 192.168.0.1:80
>         default_backend nginx
>         acl   accepts_none hdr(Accept-Encoding) -i none
>         errorfile 403 /raid/emptypage.txt
>         block if accepts_none
> 
> backend nginx
>       server srv 127.0.0.1:80 maxconn 8192
> 
> File /raid/emptypage.txt is an empty file made with 'touch 
> /raid/emptypage.txt'.
> 
> I'm doing ab2 -c 1000 -H 'Accept-Encoding: None' -n 1000000 
> http://192.168.0.1/ on two other servers and get following:
> 
> When nbproc = 1 haproxy saturates 100% of cpu core it runs at, but server is 
> running nice, I'm able to get reply from nginx behind by using curl on my 
> machine: curl http://192.168.0.1/, ab reports 16833 requests/second each and 
> longest request is around 14seconds.
> 
> When I change nbproc to higher values (maximum is 24 as there 24 threads 
> total) I can see ksoftirq/0 process saturating cpu core, network becomes slow 
> on server, ab reports same 16k-17k requests/second for each client, but 
> longest request is always around 20-30 seconds. 
> 
> I've seen such things with ksoftirq/0 running at 100% and network is almost 
> down during DDoS attacks in case of too many iptables rules but what is 
> happening now? And what number to use at nbproc? Is it ok to have haproxy 
> running at 100%? It looks like I can have 30k requests per second in my 
> setup, is there any way to make it higher? I've done some basic tuning like 
> tcp_max_tw_buckets = 1024*1024, tcp_tw_reuse = 1, tcp_max_syn_backlog = 
> 30000. Am I running out of options?

Reply via email to