Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jul 03, 2018 at 01:12:05PM -0700, Tejun Heo wrote: > Hello, Paul. > > On Tue, Jul 03, 2018 at 09:40:44AM -0700, Paul E. McKenney wrote: > > > I will apply this, but be advised that I have not seen that WARN_ON_ONCE() > > > trigger since. :-/ > > > > But I get a build error: > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jul 03, 2018 at 01:12:05PM -0700, Tejun Heo wrote: > Hello, Paul. > > On Tue, Jul 03, 2018 at 09:40:44AM -0700, Paul E. McKenney wrote: > > > I will apply this, but be advised that I have not seen that WARN_ON_ONCE() > > > trigger since. :-/ > > > > But I get a build error: > > > >

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. On Tue, Jul 03, 2018 at 09:40:44AM -0700, Paul E. McKenney wrote: > > I will apply this, but be advised that I have not seen that WARN_ON_ONCE() > > trigger since. :-/ > > But I get a build error: > > kernel/workqueue.o: In function `worker_attach_to_pool': >

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. On Tue, Jul 03, 2018 at 09:40:44AM -0700, Paul E. McKenney wrote: > > I will apply this, but be advised that I have not seen that WARN_ON_ONCE() > > trigger since. :-/ > > But I get a build error: > > kernel/workqueue.o: In function `worker_attach_to_pool': >

Re: WARN_ON_ONCE() in process_one_work()?

gt; > > I have hit this WARN_ON_ONCE() in process_one_work: > > > > > > WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && > > >raw_smp_processor_id() != pool->cpu); > > > > > > This looks like it is my rcu_gp workqueu

Re: WARN_ON_ONCE() in process_one_work()?

gt; > > I have hit this WARN_ON_ONCE() in process_one_work: > > > > > > WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && > > >raw_smp_processor_id() != pool->cpu); > > > > > > This looks like it is my rcu_gp workqueu

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, Jul 02, 2018 at 02:05:40PM -0700, Tejun Heo wrote: > Hello, Paul. > > Sorry about the late reply. > > On Wed, Jun 20, 2018 at 12:29:01PM -0700, Paul E. McKenney wrote: > > I have hit this WARN_ON_ONCE() in process_one_work: > > > > WARN_ON_ONCE(!

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, Jul 02, 2018 at 02:05:40PM -0700, Tejun Heo wrote: > Hello, Paul. > > Sorry about the late reply. > > On Wed, Jun 20, 2018 at 12:29:01PM -0700, Paul E. McKenney wrote: > > I have hit this WARN_ON_ONCE() in process_one_work: > > > > WARN_ON_ONCE(!

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. Sorry about the late reply. On Wed, Jun 20, 2018 at 12:29:01PM -0700, Paul E. McKenney wrote: > I have hit this WARN_ON_ONCE() in process_one_work: > > WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && >raw_smp_

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. Sorry about the late reply. On Wed, Jun 20, 2018 at 12:29:01PM -0700, Paul E. McKenney wrote: > I have hit this WARN_ON_ONCE() in process_one_work: > > WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && >raw_smp_

WARN_ON_ONCE() in process_one_work()?

Hello! I have hit this WARN_ON_ONCE() in process_one_work: WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); This looks like it is my rcu_gp workqueue (see splat below), and it appears to be intermitt

WARN_ON_ONCE() in process_one_work()?

Hello! I have hit this WARN_ON_ONCE() in process_one_work: WARN_ON_ONCE(!(pool->flags & POOL_DISASSOCIATED) && raw_smp_processor_id() != pool->cpu); This looks like it is my rcu_gp workqueue (see splat below), and it appears to be intermitt

Re: WARN_ON_ONCE() in process_one_work()?

On Fri, Jun 23, 2017 at 09:41:42AM -0700, Paul E. McKenney wrote: > On Wed, Jun 21, 2017 at 08:30:35AM -0700, Paul E. McKenney wrote: > > On Tue, Jun 20, 2017 at 09:45:23AM -0700, Paul E. McKenney wrote: > > > On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > > > > Hello, > > > > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Fri, Jun 23, 2017 at 09:41:42AM -0700, Paul E. McKenney wrote: > On Wed, Jun 21, 2017 at 08:30:35AM -0700, Paul E. McKenney wrote: > > On Tue, Jun 20, 2017 at 09:45:23AM -0700, Paul E. McKenney wrote: > > > On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > > > > Hello, > > > > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Wed, Jun 21, 2017 at 08:30:35AM -0700, Paul E. McKenney wrote: > On Tue, Jun 20, 2017 at 09:45:23AM -0700, Paul E. McKenney wrote: > > On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > > > Hello, > > > > > > On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Wed, Jun 21, 2017 at 08:30:35AM -0700, Paul E. McKenney wrote: > On Tue, Jun 20, 2017 at 09:45:23AM -0700, Paul E. McKenney wrote: > > On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > > > Hello, > > > > > > On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jun 20, 2017 at 09:45:23AM -0700, Paul E. McKenney wrote: > On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > > Hello, > > > > On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > > > On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > > > > Hello, > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jun 20, 2017 at 09:45:23AM -0700, Paul E. McKenney wrote: > On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > > Hello, > > > > On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > > > On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > > > > Hello, > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > Hello, > > On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > > On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > > > Hello, > > > > > > On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > >

Re: WARN_ON_ONCE() in process_one_work()?

On Sun, Jun 18, 2017 at 06:40:00AM -0400, Tejun Heo wrote: > Hello, > > On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > > On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > > > Hello, > > > > > > On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > >

Re: WARN_ON_ONCE() in process_one_work()?

Hello, On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > > Hello, > > > > On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > > > And no test failures from yesterday evening. So it looks like we get >

Re: WARN_ON_ONCE() in process_one_work()?

Hello, On Sat, Jun 17, 2017 at 10:31:05AM -0700, Paul E. McKenney wrote: > On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > > Hello, > > > > On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > > > And no test failures from yesterday evening. So it looks like we get >

Re: WARN_ON_ONCE() in process_one_work()?

On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > Hello, > > On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > > And no test failures from yesterday evening. So it looks like we get > > somewhere on the order of one failure per 138 hours of TREE07 rcutorture > >

Re: WARN_ON_ONCE() in process_one_work()?

On Sat, Jun 17, 2017 at 07:53:14AM -0400, Tejun Heo wrote: > Hello, > > On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > > And no test failures from yesterday evening. So it looks like we get > > somewhere on the order of one failure per 138 hours of TREE07 rcutorture > >

Re: WARN_ON_ONCE() in process_one_work()?

Hello, On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > And no test failures from yesterday evening. So it looks like we get > somewhere on the order of one failure per 138 hours of TREE07 rcutorture > runtime with your printk() in the mix. > > Was the above output from your

Re: WARN_ON_ONCE() in process_one_work()?

Hello, On Fri, Jun 16, 2017 at 10:36:58AM -0700, Paul E. McKenney wrote: > And no test failures from yesterday evening. So it looks like we get > somewhere on the order of one failure per 138 hours of TREE07 rcutorture > runtime with your printk() in the mix. > > Was the above output from your

Re: WARN_ON_ONCE() in process_one_work()?

On Thu, Jun 15, 2017 at 08:38:57AM -0700, Paul E. McKenney wrote: > On Wed, Jun 14, 2017 at 08:15:48AM -0700, Paul E. McKenney wrote: > > On Tue, Jun 13, 2017 at 03:31:03PM -0700, Paul E. McKenney wrote: > > > On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > > > > Hello, Paul. > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Thu, Jun 15, 2017 at 08:38:57AM -0700, Paul E. McKenney wrote: > On Wed, Jun 14, 2017 at 08:15:48AM -0700, Paul E. McKenney wrote: > > On Tue, Jun 13, 2017 at 03:31:03PM -0700, Paul E. McKenney wrote: > > > On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > > > > Hello, Paul. > > > >

Re: WARN_ON_ONCE() in process_one_work()?

On Wed, Jun 14, 2017 at 08:15:48AM -0700, Paul E. McKenney wrote: > On Tue, Jun 13, 2017 at 03:31:03PM -0700, Paul E. McKenney wrote: > > On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > > > Hello, Paul. > > > > > > On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > >

Re: WARN_ON_ONCE() in process_one_work()?

On Wed, Jun 14, 2017 at 08:15:48AM -0700, Paul E. McKenney wrote: > On Tue, Jun 13, 2017 at 03:31:03PM -0700, Paul E. McKenney wrote: > > On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > > > Hello, Paul. > > > > > > On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > >

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jun 13, 2017 at 03:31:03PM -0700, Paul E. McKenney wrote: > On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > > Hello, Paul. > > > > On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > > > Just following up... I have hit this bug a couple of times over the > >

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jun 13, 2017 at 03:31:03PM -0700, Paul E. McKenney wrote: > On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > > Hello, Paul. > > > > On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > > > Just following up... I have hit this bug a couple of times over the > >

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > Hello, Paul. > > On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > > Just following up... I have hit this bug a couple of times over the > > past few days. Anything I can do to help? > > My apologies for dropping

Re: WARN_ON_ONCE() in process_one_work()?

On Tue, Jun 13, 2017 at 04:58:37PM -0400, Tejun Heo wrote: > Hello, Paul. > > On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > > Just following up... I have hit this bug a couple of times over the > > past few days. Anything I can do to help? > > My apologies for dropping

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > Just following up... I have hit this bug a couple of times over the > past few days. Anything I can do to help? My apologies for dropping the ball on this. I've gone over the hot plug code in workqueue several

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. On Fri, May 05, 2017 at 10:11:59AM -0700, Paul E. McKenney wrote: > Just following up... I have hit this bug a couple of times over the > past few days. Anything I can do to help? My apologies for dropping the ball on this. I've gone over the hot plug code in workqueue several

Re: WARN_ON_ONCE() in process_one_work()?

-0700, Paul E. McKenney wrote: > > > > Hello! > > > > > > > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > > > > what I did wrong to make this happen: > > > > > > Oh, wait... Rescuer, it says. Might this be d

Re: WARN_ON_ONCE() in process_one_work()?

-0700, Paul E. McKenney wrote: > > > > Hello! > > > > > > > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > > > > what I did wrong to make this happen: > > > > > > Oh, wait... Rescuer, it says. Might this be d

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, 1 May 2017 15:50:58 -0400 Tejun Heo wrote: > > Were you also using nohz_full? Do you mean, "was it compiled in" or "was I actually using it. ie. had it on the command line"? It may have been compiled in, but I didn't do anything to activate it. -- Steve

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, 1 May 2017 15:50:58 -0400 Tejun Heo wrote: > > Were you also using nohz_full? Do you mean, "was it compiled in" or "was I actually using it. ie. had it on the command line"? It may have been compiled in, but I didn't do anything to activate it. -- Steve

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, May 01, 2017 at 03:42:37PM -0400, Steven Rostedt wrote: > On Mon, 1 May 2017 14:42:50 -0400 > Tejun Heo wrote: > > > Steven's involved a rescuer too. One possibility was cpuset being > > involved somehow and messing up the affinity of the rescuer kthread > >

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, May 01, 2017 at 03:42:37PM -0400, Steven Rostedt wrote: > On Mon, 1 May 2017 14:42:50 -0400 > Tejun Heo wrote: > > > Steven's involved a rescuer too. One possibility was cpuset being > > involved somehow and messing up the affinity of the rescuer kthread > > unexpectedly. Is cpuset

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, 1 May 2017 14:42:50 -0400 Tejun Heo wrote: > Steven's involved a rescuer too. One possibility was cpuset being > involved somehow and messing up the affinity of the rescuer kthread > unexpectedly. Is cpuset involved in any way? I didn't do anything with cpuset. I did

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, 1 May 2017 14:42:50 -0400 Tejun Heo wrote: > Steven's involved a rescuer too. One possibility was cpuset being > involved somehow and messing up the affinity of the rescuer kthread > unexpectedly. Is cpuset involved in any way? I didn't do anything with cpuset. I did take CPUs offline

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, May 01, 2017 at 02:44:02PM -0400, Tejun Heo wrote: > Hello, Paul. > > On Mon, May 01, 2017 at 11:38:07AM -0700, Paul E. McKenney wrote: > > On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > > > Hello! > > > > > > I am hitti

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, May 01, 2017 at 02:44:02PM -0400, Tejun Heo wrote: > Hello, Paul. > > On Mon, May 01, 2017 at 11:38:07AM -0700, Paul E. McKenney wrote: > > On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > > > Hello! > > > > > > I am hitti

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. On Mon, May 01, 2017 at 11:38:07AM -0700, Paul E. McKenney wrote: > On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > > Hello! > > > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > > what I did wrong to make

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. On Mon, May 01, 2017 at 11:38:07AM -0700, Paul E. McKenney wrote: > On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > > Hello! > > > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > > what I did wrong to make

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. Hmmm... Steven reproted a similar issue. http://lkml.kernel.org/r/20170405151628.33df7...@gandalf.local.home On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > Hello! > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > what

Re: WARN_ON_ONCE() in process_one_work()?

Hello, Paul. Hmmm... Steven reproted a similar issue. http://lkml.kernel.org/r/20170405151628.33df7...@gandalf.local.home On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > Hello! > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > what

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > Hello! > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > what I did wrong to make this happen: Oh, wait... Rescuer, it says. Might this be due to the fact that RCU's expedited grace p

Re: WARN_ON_ONCE() in process_one_work()?

On Mon, May 01, 2017 at 09:57:47AM -0700, Paul E. McKenney wrote: > Hello! > > I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering > what I did wrong to make this happen: Oh, wait... Rescuer, it says. Might this be due to the fact that RCU's expedited grace p

WARN_ON_ONCE() in process_one_work()?

Hello! I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering what I did wrong to make this happen: static void process_one_work(struct worker *worker, struct work_struct *work) __releases(>l

WARN_ON_ONCE() in process_one_work()?

Hello! I am hitting this WARN_ON_ONCE() in process_one_work() and am wondering what I did wrong to make this happen: static void process_one_work(struct worker *worker, struct work_struct *work) __releases(>l