Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Aaron Porter
On Tue, Jul 24, 2007 at 09:13:47PM +0200, Andi Kleen wrote: > You have slab debugging enabled. That makes everything slow. > It costs you ~66% of your CPU time. Disable it. Past peak for the day, but this does look like the culprit. Thanks to all who replied. - To unsubscribe from this

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread David Miller
From: Aaron Porter <[EMAIL PROTECTED]> Date: Tue, 24 Jul 2007 11:49:09 -0700 > samples %app name symbol name > 914379 48.8404 vmlinux-2.6.21.5 check_poison_obj > 341920 18.2632 vmlinux-2.6.21.5 poison_obj > 37355 1.9953 nf_conntrack

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Chris Snook
Aaron Porter wrote: On Tue, Jul 24, 2007 at 08:48:00PM +0200, Andi Kleen wrote: Aaron Porter <[EMAIL PROTECTED]> writes: I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Jeff Garzik
Turn off slab debugging and preempt, and see if that helps. Jeff - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Andi Kleen
> # opreport -l > CPU: AMD64 processors, speed 1994.52 MHz (estimated) > Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit > mask of 0x00 (No unit mask) count 10 > samples %app name symbol name > 914379 48.8404 vmlinux-2.6.21.5

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Chris Snook
Aaron Porter wrote: I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Andi Kleen
Aaron Porter <[EMAIL PROTECTED]> writes: > I'm in the process up upgrading a pool of apache servers from > 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. > Under identical network load, 2.6.21 has a load average more than 3 times > higher, cpu 0 spends well over

miserable performance of 2.6.21 under network load

2007-07-24 Thread Aaron Porter
I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under

miserable performance of 2.6.21 under network load

2007-07-24 Thread Aaron Porter
I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in interrupts (vs ~30% under

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Andi Kleen
Aaron Porter [EMAIL PROTECTED] writes: I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Chris Snook
Aaron Porter wrote: I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load, 2.6.21 has a load average more than 3 times higher, cpu 0 spends well over 90% of its time in

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Jeff Garzik
Turn off slab debugging and preempt, and see if that helps. Jeff - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Andi Kleen
# opreport -l CPU: AMD64 processors, speed 1994.52 MHz (estimated) Counted CPU_CLK_UNHALTED events (Cycles outside of halt state) with a unit mask of 0x00 (No unit mask) count 10 samples %app name symbol name 914379 48.8404 vmlinux-2.6.21.5

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Chris Snook
Aaron Porter wrote: On Tue, Jul 24, 2007 at 08:48:00PM +0200, Andi Kleen wrote: Aaron Porter [EMAIL PROTECTED] writes: I'm in the process up upgrading a pool of apache servers from 2.6.17.8 to 2.6.21.5, and we're seeing a pretty major change in behavior. Under identical network load,

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread David Miller
From: Aaron Porter [EMAIL PROTECTED] Date: Tue, 24 Jul 2007 11:49:09 -0700 samples %app name symbol name 914379 48.8404 vmlinux-2.6.21.5 check_poison_obj 341920 18.2632 vmlinux-2.6.21.5 poison_obj 37355 1.9953 nf_conntrack (no

Re: miserable performance of 2.6.21 under network load

2007-07-24 Thread Aaron Porter
On Tue, Jul 24, 2007 at 09:13:47PM +0200, Andi Kleen wrote: You have slab debugging enabled. That makes everything slow. It costs you ~66% of your CPU time. Disable it. Past peak for the day, but this does look like the culprit. Thanks to all who replied. - To unsubscribe from this