Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-10-01 Thread Linus Walleij
On Sat, Sep 22, 2012 at 11:59 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: rcu: Fix day-one dyntick-idle stall-warning bug As mentioned in another thread this solves the same problem for ux500. Reported/Tested-by: Linus Walleij linus.wall...@linaro.org But now it appears that this

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-10-01 Thread Paul E. McKenney
On Mon, Oct 01, 2012 at 10:55:11AM +0200, Linus Walleij wrote: On Sat, Sep 22, 2012 at 11:59 PM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: rcu: Fix day-one dyntick-idle stall-warning bug As mentioned in another thread this solves the same problem for ux500. Reported/Tested-by:

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-24 Thread Shilimkar, Santosh
On Sun, Sep 23, 2012 at 3:29 AM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: On Sat, Sep 22, 2012 at 01:10:43PM -0700, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 06:42:08PM +, Paul Walmsley wrote: On Fri, 21 Sep 2012, Paul E. McKenney wrote: [...] And here is a patch. I am

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-24 Thread Paul E. McKenney
On Mon, Sep 24, 2012 at 03:11:34PM +0530, Shilimkar, Santosh wrote: On Sun, Sep 23, 2012 at 3:29 AM, Paul E. McKenney paul...@linux.vnet.ibm.com wrote: On Sat, Sep 22, 2012 at 01:10:43PM -0700, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 06:42:08PM +, Paul Walmsley wrote: On Fri,

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-24 Thread Paul Walmsley
On Sat, 22 Sep 2012, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 10:20:19PM +, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: This thing has been in the kernel since about 2004, not sure why you didn't hit it earlier. One other data point in that regard

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-24 Thread Paul E. McKenney
On Mon, Sep 24, 2012 at 09:54:00PM +, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 10:20:19PM +, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: This thing has been in the kernel since about 2004, not sure why

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-23 Thread Paul Walmsley
On Sat, 22 Sep 2012, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 10:25:59PM +, Paul Walmsley wrote: The recent tests here have been on Pandaboard, which is dual-CPU, but my recollection is that I also observed the warnings on a single-core Beagleboard. Will re-test.

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-23 Thread Paul E. McKenney
On Sun, Sep 23, 2012 at 07:55:50AM +, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 10:25:59PM +, Paul Walmsley wrote: The recent tests here have been on Pandaboard, which is dual-CPU, but my recollection is that I also observed

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Frederic Weisbecker
2012/9/22 Paul E. McKenney paul...@linux.vnet.ibm.com: On Fri, Sep 21, 2012 at 01:31:49PM -0700, Tony Lindgren wrote: * Paul E. McKenney paul...@linux.vnet.ibm.com [120921 12:58]: Just to make sure I understand the combinations: o All stalls have happened when running a minimal

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sat, Sep 22, 2012 at 05:45:12PM +0200, Frederic Weisbecker wrote: 2012/9/22 Paul E. McKenney paul...@linux.vnet.ibm.com: On Fri, Sep 21, 2012 at 01:31:49PM -0700, Tony Lindgren wrote: * Paul E. McKenney paul...@linux.vnet.ibm.com [120921 12:58]: Just to make sure I understand the

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul Walmsley
Hi Paul On Fri, 21 Sep 2012, Paul E. McKenney wrote: I am wondering if your system somehow figured out how to start a grace period that had no RCU callbacks waiting for it. If that happened, then a CONFIG_NO_HZ=y system could in theory get into a state where all CPUs are in dyntick-idle

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul Walmsley
On Fri, 21 Sep 2012, Paul E. McKenney wrote: Could you please point me to a recipe for creating a minimal userspace? Just in case it is the userspac erather than the architecture/hardware that makes the difference. Tony's suggestion is pretty good. Note that there may also be differences in

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sat, Sep 22, 2012 at 06:42:08PM +, Paul Walmsley wrote: On Fri, 21 Sep 2012, Paul E. McKenney wrote: Could you please point me to a recipe for creating a minimal userspace? Just in case it is the userspac erather than the architecture/hardware that makes the difference. Tony's

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sat, Sep 22, 2012 at 06:16:15PM +, Paul Walmsley wrote: Hi Paul On Fri, 21 Sep 2012, Paul E. McKenney wrote: I am wondering if your system somehow figured out how to start a grace period that had no RCU callbacks waiting for it. If that happened, then a CONFIG_NO_HZ=y system

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sat, Sep 22, 2012 at 01:10:43PM -0700, Paul E. McKenney wrote: On Sat, Sep 22, 2012 at 06:42:08PM +, Paul Walmsley wrote: On Fri, 21 Sep 2012, Paul E. McKenney wrote: Could you please point me to a recipe for creating a minimal userspace? Just in case it is the userspac erather

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul Walmsley
Hi Paul On Sat, 22 Sep 2012, Paul E. McKenney wrote: Strangely enough, I believe that I have inadvertently fixed this in my -rcu tree: git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git rcu/next Nevertheless, if you get a chance to try it, I would be interested to hear

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul Walmsley
On Sat, 22 Sep 2012, Paul E. McKenney wrote: And here is a patch. I am still having trouble reproducing the problem, but figured that I should avoid serializing things. Thanks, testing this now on v3.6-rc6. One question though about the patch description: All this begs the question of

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sat, Sep 22, 2012 at 10:25:59PM +, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: And here is a patch. I am still having trouble reproducing the problem, but figured that I should avoid serializing things. Thanks, testing this now on v3.6-rc6. Very cool, thank

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sat, Sep 22, 2012 at 10:20:19PM +, Paul Walmsley wrote: Hi Paul On Sat, 22 Sep 2012, Paul E. McKenney wrote: Strangely enough, I believe that I have inadvertently fixed this in my -rcu tree: git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git rcu/next

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul Walmsley
Hi Paul On Sat, 22 Sep 2012, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: And here is a patch. I am still having trouble reproducing the problem, but figured that I should avoid serializing things. Thanks, testing this now on v3.6-rc6. Looks like you solved it!

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul E. McKenney
On Sun, Sep 23, 2012 at 01:42:10AM +, Paul Walmsley wrote: Hi Paul On Sat, 22 Sep 2012, Paul Walmsley wrote: On Sat, 22 Sep 2012, Paul E. McKenney wrote: And here is a patch. I am still having trouble reproducing the problem, but figured that I should avoid serializing

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-22 Thread Paul Walmsley
On Sat, 22 Sep 2012, Paul E. McKenney wrote: Very cool, thank you for your testing efforts!!! You're welcome. May I apply your Tested-by to this patch? Please do: Tested-by: Paul Walmsley p...@pwsan.com # OMAP4430 Am testing on OMAP3730 (single-core) now. - Paul -- To unsubscribe from

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul Walmsley
Hi Paul On Thu, 20 Sep 2012, Paul Walmsley wrote: On Thu, 20 Sep 2012, Paul E. McKenney wrote: Paul Walmsley, please let me know if the config below doesn't clear things up for you or if there is some reason why this config is infeasible. Will certainly test it here once things settle

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul Walmsley
On Fri, 21 Sep 2012, Paul Walmsley wrote: The config used was 'omap2plus_defconfig', and enabled CONFIG_CPU_IDLE by hand. One other thing I forgot to mention - CONFIG_RCU_CPU_STALL_INFO was enabled by hand also. Below is the diff between omap2plus_defconfig and the config that was used

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul Walmsley
cc Frederic Weisbecker - context is here: http://marc.info/?l=linux-kernelm=134749030206016w=2 On Thu, 20 Sep 2012, Paul E. McKenney wrote: Fair point. I am wondering whether there is some path into the idle loop that somehow avoids telling RCU that the CPU has in face entered idle.

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul E. McKenney
On Fri, Sep 21, 2012 at 06:08:59PM +, Paul Walmsley wrote: cc Frederic Weisbecker - context is here: http://marc.info/?l=linux-kernelm=134749030206016w=2 On Thu, 20 Sep 2012, Paul E. McKenney wrote: Fair point. I am wondering whether there is some path into the idle loop that

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul Walmsley
Hi Just did a test with CONFIG_NO_HZ=n, and no rcu_sched stall messages appeared for 60 minutes. Following is the diff from omap2plus_defconfig. - Paul --- .config 2012-09-21 12:51:19.114938954 -0600 +++ .testconfig 2012-09-21 12:51:15.686926318 -0600 @@ -69,7 +69,7 @@ # Timers subsystem

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul Walmsley
On Fri, 21 Sep 2012, Paul E. McKenney wrote: On Fri, Sep 21, 2012 at 06:08:59PM +, Paul Walmsley wrote: As far as I know, our only idle entry point is in arch/arm/common/process.c:cpu_idle(). In mainline, this is arch/arm/kernel/process.c, correct? Indeed; sorry about that,

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul E. McKenney
On Fri, Sep 21, 2012 at 07:11:14PM +, Paul Walmsley wrote: On Fri, 21 Sep 2012, Paul E. McKenney wrote: On Fri, Sep 21, 2012 at 06:08:59PM +, Paul Walmsley wrote: As far as I know, our only idle entry point is in arch/arm/common/process.c:cpu_idle(). In mainline, this

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Tony Lindgren
* Paul E. McKenney paul...@linux.vnet.ibm.com [120921 12:58]: Just to make sure I understand the combinations: o All stalls have happened when running a minimal userspace. o CONFIG_NO_HZ=n suppresses the stalls. o CONFIG_RCU_FAST_NO_HZ (which depends on CONFIG_NO_HZ=y) has

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul E. McKenney
On Fri, Sep 21, 2012 at 05:47:31PM +, Paul Walmsley wrote: Hi Paul On Thu, 20 Sep 2012, Paul Walmsley wrote: On Thu, 20 Sep 2012, Paul E. McKenney wrote: Paul Walmsley, please let me know if the config below doesn't clear things up for you or if there is some reason why this

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul E. McKenney
On Fri, Sep 21, 2012 at 01:31:49PM -0700, Tony Lindgren wrote: * Paul E. McKenney paul...@linux.vnet.ibm.com [120921 12:58]: Just to make sure I understand the combinations: o All stalls have happened when running a minimal userspace. o CONFIG_NO_HZ=n suppresses the stalls. o

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul E. McKenney
On Fri, Sep 21, 2012 at 12:57:17PM -0700, Paul E. McKenney wrote: On Fri, Sep 21, 2012 at 07:11:14PM +, Paul Walmsley wrote: On Fri, 21 Sep 2012, Paul E. McKenney wrote: [ . . . ] I may take your advice of remote access to a Panda board, though that is likely to take a bit of time

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul Walmsley
On Fri, 21 Sep 2012, Paul E. McKenney wrote: On Fri, Sep 21, 2012 at 05:47:31PM +, Paul Walmsley wrote: I built an OMAP kernel from Linus' commit 4651afbbae968772efd6dc4ba461cba9b49bb9d8 (Merge branch 'for-3.6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq). The

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-21 Thread Paul E. McKenney
On Fri, Sep 21, 2012 at 10:41:14PM +, Paul Walmsley wrote: On Fri, 21 Sep 2012, Paul E. McKenney wrote: On Fri, Sep 21, 2012 at 05:47:31PM +, Paul Walmsley wrote: I built an OMAP kernel from Linus' commit 4651afbbae968772efd6dc4ba461cba9b49bb9d8 (Merge branch 'for-3.6-fixes'

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-20 Thread Paul Walmsley
Hi, On Wed, 19 Sep 2012, Paul E. McKenney wrote: On Thu, Sep 13, 2012 at 06:52:10PM +, Paul Walmsley wrote: On Wed, 12 Sep 2012, Paul E. McKenney wrote: Subodh Nijsure (also CCed) reported something that might be similar on ARM, and also reported that setting the following got

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-20 Thread Bruce, Becky
I'll give it a try on my panda later today. B Sent from my iPhone On Sep 20, 2012, at 2:56 AM, Paul Walmsley p...@pwsan.com wrote: Hi, On Wed, 19 Sep 2012, Paul E. McKenney wrote: On Thu, Sep 13, 2012 at 06:52:10PM +, Paul Walmsley wrote: On Wed, 12 Sep 2012, Paul E. McKenney

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-20 Thread Bruce, Becky
On Sep 20, 2012, at 2:56 AM, Paul Walmsley wrote: Hi, On Wed, 19 Sep 2012, Paul E. McKenney wrote: On Thu, Sep 13, 2012 at 06:52:10PM +, Paul Walmsley wrote: On Wed, 12 Sep 2012, Paul E. McKenney wrote: Subodh Nijsure (also CCed) reported something that might be similar on ARM,

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-20 Thread Paul E. McKenney
On Thu, Sep 20, 2012 at 09:49:13PM +, Bruce, Becky wrote: On Sep 20, 2012, at 2:56 AM, Paul Walmsley wrote: Hi, On Wed, 19 Sep 2012, Paul E. McKenney wrote: On Thu, Sep 13, 2012 at 06:52:10PM +, Paul Walmsley wrote: On Wed, 12 Sep 2012, Paul E. McKenney wrote:

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-20 Thread Paul Walmsley
On Thu, 20 Sep 2012, Paul E. McKenney wrote: On Thu, Sep 20, 2012 at 09:49:13PM +, Bruce, Becky wrote: OK, people, you can stop heckling me about sent from my iPhone - I'm in the wilds of rural Louisiana with really bad internet service and was trying to work on my phone (but, alas,

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-20 Thread Paul E. McKenney
On Thu, Sep 20, 2012 at 10:47:25PM +, Paul Walmsley wrote: On Thu, 20 Sep 2012, Paul E. McKenney wrote: On Thu, Sep 20, 2012 at 09:49:13PM +, Bruce, Becky wrote: OK, people, you can stop heckling me about sent from my iPhone - I'm in the wilds of rural Louisiana with really

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-19 Thread Paul E. McKenney
On Thu, Sep 13, 2012 at 06:52:10PM +, Paul Walmsley wrote: Hi Paul, thanks for the reply, On Wed, 12 Sep 2012, Paul E. McKenney wrote: Interesting. I am assuming that the interrupt in the stack below came from idle, if not, please let me know what. According to the exception

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-13 Thread Paul Walmsley
Hi Paul, thanks for the reply, On Wed, 12 Sep 2012, Paul E. McKenney wrote: Interesting. I am assuming that the interrupt in the stack below came from idle, if not, please let me know what. According to the exception stack section in the original traceback, it appears that the serial

rcu self-detected stall messages on OMAP3, 4 boards

2012-09-12 Thread Paul Walmsley
Hi Paul Recently several of us have been seeing INFO: rcu_sched self-detected stall on CPU { 0} (t=20611 jiffies) stack tracebacks on various OMAP3 and 4 board. I only noticed it during v3.6-rc3, but I suspect it's been happening for users at least since May:

Re: rcu self-detected stall messages on OMAP3, 4 boards

2012-09-12 Thread Paul E. McKenney
On Wed, Sep 12, 2012 at 10:51:30PM +, Paul Walmsley wrote: Hi Paul Recently several of us have been seeing INFO: rcu_sched self-detected stall on CPU { 0} (t=20611 jiffies) stack tracebacks on various OMAP3 and 4 board. I only noticed it during v3.6-rc3, but I suspect it's been