Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-24 Thread Paul E. McKenney
On Tue, Jun 24, 2014 at 01:43:16PM -0700, Dave Hansen wrote: > On 06/23/2014 05:39 PM, Paul E. McKenney wrote: > > On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: > >> On 06/23/2014 05:15 PM, Paul E. McKenney wrote: > >>> Just out of curiosity, how many CPUs does your system have?

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-24 Thread Dave Hansen
On 06/23/2014 05:39 PM, Paul E. McKenney wrote: > On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: >> On 06/23/2014 05:15 PM, Paul E. McKenney wrote: >>> Just out of curiosity, how many CPUs does your system have? 80? >>> If 160, looks like something bad is happening at 80. >> >> 80

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-24 Thread Dave Hansen
On 06/23/2014 05:39 PM, Paul E. McKenney wrote: > On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: >> On 06/23/2014 05:15 PM, Paul E. McKenney wrote: >>> Just out of curiosity, how many CPUs does your system have? 80? >>> If 160, looks like something bad is happening at 80. >> >> 80

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-24 Thread Dave Hansen
On 06/23/2014 05:39 PM, Paul E. McKenney wrote: On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: On 06/23/2014 05:15 PM, Paul E. McKenney wrote: Just out of curiosity, how many CPUs does your system have? 80? If 160, looks like something bad is happening at 80. 80 cores, 160

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-24 Thread Dave Hansen
On 06/23/2014 05:39 PM, Paul E. McKenney wrote: On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: On 06/23/2014 05:15 PM, Paul E. McKenney wrote: Just out of curiosity, how many CPUs does your system have? 80? If 160, looks like something bad is happening at 80. 80 cores, 160

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-24 Thread Paul E. McKenney
On Tue, Jun 24, 2014 at 01:43:16PM -0700, Dave Hansen wrote: On 06/23/2014 05:39 PM, Paul E. McKenney wrote: On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: On 06/23/2014 05:15 PM, Paul E. McKenney wrote: Just out of curiosity, how many CPUs does your system have? 80? If

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: > On 06/23/2014 05:15 PM, Paul E. McKenney wrote: > > Just out of curiosity, how many CPUs does your system have? 80? > > If 160, looks like something bad is happening at 80. > > 80 cores, 160 threads. >80 processes/threads is where

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 05:15 PM, Paul E. McKenney wrote: > Just out of curiosity, how many CPUs does your system have? 80? > If 160, looks like something bad is happening at 80. 80 cores, 160 threads. >80 processes/threads is where we start using the second thread on the cores. The tasks are also

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 04:30:12PM -0700, Dave Hansen wrote: > On 06/23/2014 11:09 AM, Paul E. McKenney wrote: > > So let's see... The open1 benchmark sits in a loop doing open() > > and close(), and probably spends most of its time in the kernel. > > It doesn't do much context switching. I am

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 11:09 AM, Paul E. McKenney wrote: > So let's see... The open1 benchmark sits in a loop doing open() > and close(), and probably spends most of its time in the kernel. > It doesn't do much context switching. I am guessing that you don't > have CONFIG_NO_HZ_FULL=y, or the boot/sysfs

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 10:17:19AM -0700, Dave Hansen wrote: > On 06/23/2014 09:55 AM, Dave Hansen wrote: > > This still has a regression. Commit 1ed70de (from Paul's git tree), > > gets a result of 52231880. If I back up two commits to v3.16-rc1 and > > revert ac1bea85 (the original culprit)

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 10:19:05AM -0700, Andi Kleen wrote: > > In 3.10, RCU had 14,046 lines of code, not counting documentation and > > test scripting. In 3.15, RCU had 13,208 lines of code, again not counting > > documentation and test scripting. That is a decrease of almost 1KLoC, > > so

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 10:16 AM, Paul E. McKenney wrote: > On Mon, Jun 23, 2014 at 09:55:21AM -0700, Dave Hansen wrote: >> This still has a regression. Commit 1ed70de (from Paul's git tree), >> gets a result of 52231880. If I back up two commits to v3.16-rc1 and >> revert ac1bea85 (the original culprit)

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Andi Kleen
> In 3.10, RCU had 14,046 lines of code, not counting documentation and > test scripting. In 3.15, RCU had 13,208 lines of code, again not counting > documentation and test scripting. That is a decrease of almost 1KLoC, > so your wish is granted. Ok that's good progress. > CONFIG_RCU_NOCB_CPU,

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 09:55 AM, Dave Hansen wrote: > This still has a regression. Commit 1ed70de (from Paul's git tree), > gets a result of 52231880. If I back up two commits to v3.16-rc1 and > revert ac1bea85 (the original culprit) the result goes back up to 57308512. > > So something is still going

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 09:55:21AM -0700, Dave Hansen wrote: > This still has a regression. Commit 1ed70de (from Paul's git tree), > gets a result of 52231880. If I back up two commits to v3.16-rc1 and > revert ac1bea85 (the original culprit) the result goes back up to 57308512. > > So

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
This still has a regression. Commit 1ed70de (from Paul's git tree), gets a result of 52231880. If I back up two commits to v3.16-rc1 and revert ac1bea85 (the original culprit) the result goes back up to 57308512. So something is still going on here. I'll go back and compare the grace period

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:51:08AM -0500, Christoph Lameter wrote: > On Mon, 23 Jun 2014, Peter Zijlstra wrote: > > > On the topic of these threads; I recently noticed RCU grew a metric ton > > of them, I found some 75 rcu kthreads on my box, wth up with that? > > Would kworker threads work for

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:49:31AM -0700, Andi Kleen wrote: > > On the topic of these threads; I recently noticed RCU grew a metric ton > > of them, I found some 75 rcu kthreads on my box, wth up with that? > > It seems like RCU is growing in complexity all the time. > > Can it be put on a diet

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Andi Kleen
> On the topic of these threads; I recently noticed RCU grew a metric ton > of them, I found some 75 rcu kthreads on my box, wth up with that? It seems like RCU is growing in complexity all the time. Can it be put on a diet in general? No more new CONFIGs please either. -Andi --

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:53:12AM -0500, Christoph Lameter wrote: > On Fri, 20 Jun 2014, Paul E. McKenney wrote: > > > > I like this approach *far* better. This is the kind of thing I had in > > > mind when I suggested using the fqs machinery: remove the poll entirely > > > and just thwack a

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:26:15AM +0200, Peter Zijlstra wrote: > On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: > > Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) > > fixed a problem where a CPU looping in the kernel with but one runnable > > task would

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Christoph Lameter
On Fri, 20 Jun 2014, Paul E. McKenney wrote: > > I like this approach *far* better. This is the kind of thing I had in > > mind when I suggested using the fqs machinery: remove the poll entirely > > and just thwack a CPU if it takes too long without a quiescent state. > > Reviewed-by: Josh

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Christoph Lameter
On Mon, 23 Jun 2014, Peter Zijlstra wrote: > On the topic of these threads; I recently noticed RCU grew a metric ton > of them, I found some 75 rcu kthreads on my box, wth up with that? Would kworker threads work for rcu? That would also avoid the shifting around of RCU threads for NOHZ

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Peter Zijlstra
On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: > Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) > fixed a problem where a CPU looping in the kernel with but one runnable > task would give RCU CPU stall warnings, even if the in-kernel loop > contained

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Christoph Lameter
On Mon, 23 Jun 2014, Peter Zijlstra wrote: On the topic of these threads; I recently noticed RCU grew a metric ton of them, I found some 75 rcu kthreads on my box, wth up with that? Would kworker threads work for rcu? That would also avoid the shifting around of RCU threads for NOHZ

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Christoph Lameter
On Fri, 20 Jun 2014, Paul E. McKenney wrote: I like this approach *far* better. This is the kind of thing I had in mind when I suggested using the fqs machinery: remove the poll entirely and just thwack a CPU if it takes too long without a quiescent state. Reviewed-by: Josh Triplett

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:26:15AM +0200, Peter Zijlstra wrote: On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) fixed a problem where a CPU looping in the kernel with but one runnable task would give RCU

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:53:12AM -0500, Christoph Lameter wrote: On Fri, 20 Jun 2014, Paul E. McKenney wrote: I like this approach *far* better. This is the kind of thing I had in mind when I suggested using the fqs machinery: remove the poll entirely and just thwack a CPU if it

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Andi Kleen
On the topic of these threads; I recently noticed RCU grew a metric ton of them, I found some 75 rcu kthreads on my box, wth up with that? It seems like RCU is growing in complexity all the time. Can it be put on a diet in general? No more new CONFIGs please either. -Andi --

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:49:31AM -0700, Andi Kleen wrote: On the topic of these threads; I recently noticed RCU grew a metric ton of them, I found some 75 rcu kthreads on my box, wth up with that? It seems like RCU is growing in complexity all the time. Can it be put on a diet in

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 08:51:08AM -0500, Christoph Lameter wrote: On Mon, 23 Jun 2014, Peter Zijlstra wrote: On the topic of these threads; I recently noticed RCU grew a metric ton of them, I found some 75 rcu kthreads on my box, wth up with that? Would kworker threads work for rcu?

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
This still has a regression. Commit 1ed70de (from Paul's git tree), gets a result of 52231880. If I back up two commits to v3.16-rc1 and revert ac1bea85 (the original culprit) the result goes back up to 57308512. So something is still going on here. I'll go back and compare the grace period

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 09:55:21AM -0700, Dave Hansen wrote: This still has a regression. Commit 1ed70de (from Paul's git tree), gets a result of 52231880. If I back up two commits to v3.16-rc1 and revert ac1bea85 (the original culprit) the result goes back up to 57308512. So something is

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 09:55 AM, Dave Hansen wrote: This still has a regression. Commit 1ed70de (from Paul's git tree), gets a result of 52231880. If I back up two commits to v3.16-rc1 and revert ac1bea85 (the original culprit) the result goes back up to 57308512. So something is still going on

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Andi Kleen
In 3.10, RCU had 14,046 lines of code, not counting documentation and test scripting. In 3.15, RCU had 13,208 lines of code, again not counting documentation and test scripting. That is a decrease of almost 1KLoC, so your wish is granted. Ok that's good progress. CONFIG_RCU_NOCB_CPU,

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 10:16 AM, Paul E. McKenney wrote: On Mon, Jun 23, 2014 at 09:55:21AM -0700, Dave Hansen wrote: This still has a regression. Commit 1ed70de (from Paul's git tree), gets a result of 52231880. If I back up two commits to v3.16-rc1 and revert ac1bea85 (the original culprit) the

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 10:19:05AM -0700, Andi Kleen wrote: In 3.10, RCU had 14,046 lines of code, not counting documentation and test scripting. In 3.15, RCU had 13,208 lines of code, again not counting documentation and test scripting. That is a decrease of almost 1KLoC, so your wish

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 10:17:19AM -0700, Dave Hansen wrote: On 06/23/2014 09:55 AM, Dave Hansen wrote: This still has a regression. Commit 1ed70de (from Paul's git tree), gets a result of 52231880. If I back up two commits to v3.16-rc1 and revert ac1bea85 (the original culprit) the

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 11:09 AM, Paul E. McKenney wrote: So let's see... The open1 benchmark sits in a loop doing open() and close(), and probably spends most of its time in the kernel. It doesn't do much context switching. I am guessing that you don't have CONFIG_NO_HZ_FULL=y, or the boot/sysfs

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 04:30:12PM -0700, Dave Hansen wrote: On 06/23/2014 11:09 AM, Paul E. McKenney wrote: So let's see... The open1 benchmark sits in a loop doing open() and close(), and probably spends most of its time in the kernel. It doesn't do much context switching. I am guessing

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Dave Hansen
On 06/23/2014 05:15 PM, Paul E. McKenney wrote: Just out of curiosity, how many CPUs does your system have? 80? If 160, looks like something bad is happening at 80. 80 cores, 160 threads. 80 processes/threads is where we start using the second thread on the cores. The tasks are also pinned

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Paul E. McKenney
On Mon, Jun 23, 2014 at 05:20:30PM -0700, Dave Hansen wrote: On 06/23/2014 05:15 PM, Paul E. McKenney wrote: Just out of curiosity, how many CPUs does your system have? 80? If 160, looks like something bad is happening at 80. 80 cores, 160 threads. 80 processes/threads is where we start

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-23 Thread Peter Zijlstra
On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) fixed a problem where a CPU looping in the kernel with but one runnable task would give RCU CPU stall warnings, even if the in-kernel loop contained

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-21 Thread Paul E. McKenney
On Fri, Jun 20, 2014 at 09:29:58PM -0700, Josh Triplett wrote: > On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: > > Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) > > fixed a problem where a CPU looping in the kernel with but one runnable > > task would

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-21 Thread Paul E. McKenney
On Fri, Jun 20, 2014 at 09:29:58PM -0700, Josh Triplett wrote: On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) fixed a problem where a CPU looping in the kernel with but one runnable task would give RCU

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-20 Thread Josh Triplett
On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: > Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) > fixed a problem where a CPU looping in the kernel with but one runnable > task would give RCU CPU stall warnings, even if the in-kernel loop > contained

Re: [PATCH tip/core/rcu] Reduce overhead of cond_resched() checks for RCU

2014-06-20 Thread Josh Triplett
On Fri, Jun 20, 2014 at 07:59:58PM -0700, Paul E. McKenney wrote: Commit ac1bea85781e (Make cond_resched() report RCU quiescent states) fixed a problem where a CPU looping in the kernel with but one runnable task would give RCU CPU stall warnings, even if the in-kernel loop contained