Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-18 Thread Paul E. McKenney
On Sun, Jan 13, 2013 at 04:13:51AM +, Ben Hutchings wrote:
> On Wed, 2013-01-02 at 19:49 -0500, Steven Rostedt wrote:
> > On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
> > > On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
> > > > On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
> > > > > This happened of a virtual guest (Linux mail 3.2.0-35-virtual 
> > > > > #55-Ubuntu SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 
> > > > > GNU/Linux)
> > > > 
> > > > 3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
> > > > to file a Bugzilla (or whatever they call it) with them.
> > > > 
> > > > > 
> > > > > Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected 
> > > > > stall on CPU 3 (t=0 jiffies)
> > > > 
> > > > 0 jiffies?
> > > 
> > > One possibility is that they are missing commit a10d206e (rcu: Fix
> > > day-one dyntick-idle stall-warning bug), at least assuming that
> > > CONFIG_PREEMPT=n.  This went into 3.6, IIRC.
> > 
> > If that does fix the issue, we probably should send that off to Ben for
> > inclusion into 3.2 stable.
> 
> If I read the Ubuntu changelog correctly, Ralf's machine is running a
> version based on 3.2.34, which already has this change (I applied it in
> 3.2.32).

OK, another possibility is c8020a67.  Ralf, if that does not help, is
it possible for you to run either a 3.7 or a -rcu kernel [*]?  There are
some other fixes that could be related, but the probability seems low.

Thanx, Paul

[*]  git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
branch rcu/next

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-18 Thread Paul E. McKenney
On Sun, Jan 13, 2013 at 04:13:51AM +, Ben Hutchings wrote:
 On Wed, 2013-01-02 at 19:49 -0500, Steven Rostedt wrote:
  On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
   On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
 This happened of a virtual guest (Linux mail 3.2.0-35-virtual 
 #55-Ubuntu SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 
 GNU/Linux)

3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
to file a Bugzilla (or whatever they call it) with them.

 
 Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected 
 stall on CPU 3 (t=0 jiffies)

0 jiffies?
   
   One possibility is that they are missing commit a10d206e (rcu: Fix
   day-one dyntick-idle stall-warning bug), at least assuming that
   CONFIG_PREEMPT=n.  This went into 3.6, IIRC.
  
  If that does fix the issue, we probably should send that off to Ben for
  inclusion into 3.2 stable.
 
 If I read the Ubuntu changelog correctly, Ralf's machine is running a
 version based on 3.2.34, which already has this change (I applied it in
 3.2.32).

OK, another possibility is c8020a67.  Ralf, if that does not help, is
it possible for you to run either a 3.7 or a -rcu kernel [*]?  There are
some other fixes that could be related, but the probability seems low.

Thanx, Paul

[*]  git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git
branch rcu/next

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-12 Thread Ben Hutchings
On Wed, 2013-01-02 at 19:49 -0500, Steven Rostedt wrote:
> On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
> > On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
> > > On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
> > > > This happened of a virtual guest (Linux mail 3.2.0-35-virtual 
> > > > #55-Ubuntu SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 
> > > > GNU/Linux)
> > > 
> > > 3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
> > > to file a Bugzilla (or whatever they call it) with them.
> > > 
> > > > 
> > > > Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected 
> > > > stall on CPU 3 (t=0 jiffies)
> > > 
> > > 0 jiffies?
> > 
> > One possibility is that they are missing commit a10d206e (rcu: Fix
> > day-one dyntick-idle stall-warning bug), at least assuming that
> > CONFIG_PREEMPT=n.  This went into 3.6, IIRC.
> 
> If that does fix the issue, we probably should send that off to Ben for
> inclusion into 3.2 stable.

If I read the Ubuntu changelog correctly, Ralf's machine is running a
version based on 3.2.34, which already has this change (I applied it in
3.2.32).

Ben.

-- 
Ben Hutchings
Klipstein's 4th Law of Prototyping and Production:
A fail-safe circuit will destroy others.


signature.asc
Description: This is a digitally signed message part


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-12 Thread Ben Hutchings
On Wed, 2013-01-02 at 19:49 -0500, Steven Rostedt wrote:
 On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
  On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
   On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
This happened of a virtual guest (Linux mail 3.2.0-35-virtual 
#55-Ubuntu SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 
GNU/Linux)
   
   3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
   to file a Bugzilla (or whatever they call it) with them.
   

Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected 
stall on CPU 3 (t=0 jiffies)
   
   0 jiffies?
  
  One possibility is that they are missing commit a10d206e (rcu: Fix
  day-one dyntick-idle stall-warning bug), at least assuming that
  CONFIG_PREEMPT=n.  This went into 3.6, IIRC.
 
 If that does fix the issue, we probably should send that off to Ben for
 inclusion into 3.2 stable.

If I read the Ubuntu changelog correctly, Ralf's machine is running a
version based on 3.2.34, which already has this change (I applied it in
3.2.32).

Ben.

-- 
Ben Hutchings
Klipstein's 4th Law of Prototyping and Production:
A fail-safe circuit will destroy others.


signature.asc
Description: This is a digitally signed message part


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Paul E. McKenney
On Wed, Jan 02, 2013 at 07:49:23PM -0500, Steven Rostedt wrote:
> On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
> > On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
> > > On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
> > > > This happened of a virtual guest (Linux mail 3.2.0-35-virtual 
> > > > #55-Ubuntu SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 
> > > > GNU/Linux)
> > > 
> > > 3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
> > > to file a Bugzilla (or whatever they call it) with them.
> > > 
> > > > 
> > > > Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected 
> > > > stall on CPU 3 (t=0 jiffies)
> > > 
> > > 0 jiffies?
> > 
> > One possibility is that they are missing commit a10d206e (rcu: Fix
> > day-one dyntick-idle stall-warning bug), at least assuming that
> > CONFIG_PREEMPT=n.  This went into 3.6, IIRC.
> 
> If that does fix the issue, we probably should send that off to Ben for
> inclusion into 3.2 stable.

Looks like you already CCed him, thank you!  ;-)

Thanx, Paul

> Thanks,
> 
> -- Steve
> 
> > 
> > Thanx, Paul
> > 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Steven Rostedt
On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
> On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
> > On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
> > > This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu 
> > > SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)
> > 
> > 3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
> > to file a Bugzilla (or whatever they call it) with them.
> > 
> > > 
> > > Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall 
> > > on CPU 3 (t=0 jiffies)
> > 
> > 0 jiffies?
> 
> One possibility is that they are missing commit a10d206e (rcu: Fix
> day-one dyntick-idle stall-warning bug), at least assuming that
> CONFIG_PREEMPT=n.  This went into 3.6, IIRC.

If that does fix the issue, we probably should send that off to Ben for
inclusion into 3.2 stable.

Thanks,

-- Steve

> 
>   Thanx, Paul
> 



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Paul E. McKenney
On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
> On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
> > This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu 
> > SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)
> 
> 3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
> to file a Bugzilla (or whatever they call it) with them.
> 
> > 
> > Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall 
> > on CPU 3 (t=0 jiffies)
> 
> 0 jiffies?

One possibility is that they are missing commit a10d206e (rcu: Fix
day-one dyntick-idle stall-warning bug), at least assuming that
CONFIG_PREEMPT=n.  This went into 3.6, IIRC.

Thanx, Paul

> > Jan  2 13:24:08 mail kernel: [3531872.484398] sending NMI to all CPUs:
> > Jan  2 13:24:08 mail kernel: [3531872.484430] NMI backtrace for cpu 3
> > Jan  2 13:24:08 mail kernel: [3531872.484432] CPU 3
> > Jan  2 13:24:08 mail kernel: [3531872.484433] Modules linked in: btrfs 
> > zlib_deflate libcrc32c ufs vfat msdos fat xfs ip6_queue 
> > nf_conntrack_proto_udplite xt_TCPMSS xt_sctp ip6t_LOG ip6t_REJECT 
> > nf_conntrack_ipv6 ip6table_raw ip6table_mangle ip6table_filter act_police 
> > cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb sch_hfsc 
> > sch_ingress sch_sfq xt_time xt_connlimit xt_realm xt_addrtype iptable_raw 
> > xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP 
> > ipt_MASQUERADE ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah xt_set ip_set 
> > nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp 
> > nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp 
> > nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip 
> > nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre 
> > nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast 
> > nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_TPROXY 
> > nf_tproxy_core ip6_tables nf_defrag_i
> >  pv6 xt_tcpmss xt_pkttype xt
> > Jan  2 13:24:08 mail kernel: _physdev xt_owner xt_NFQUEUE xt_NFLOG 
> > nfnetlink_log xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange 
> > xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_connmark 
> > xt_CLASSIFY xt_AUDIT ipt_LOG xt_tcpudp xt_state iptable_nat nf_nat 
> > nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink 
> > iptable_filter ip_tables x_tables ext2 psmouse serio_raw virtio_balloon lp 
> > parport floppy
> > Jan  2 13:24:08 mail kernel: [3531872.484533]
> > Jan  2 13:24:08 mail kernel: [3531872.484536] Pid: 0, comm: swapper/3 Not 
> > tainted 3.2.0-33-virtual #52-Ubuntu Bochs Bochs
> > Jan  2 13:24:08 mail kernel: [3531872.484539] RIP: 
> > 0010:[]  [] flat_send_IPI_all+0xaf/0xd0
> > Jan  2 13:24:08 mail kernel: [3531872.484548] RSP: 0018:88011fd83d88  
> > EFLAGS: 00010006
> > Jan  2 13:24:08 mail kernel: [3531872.484550] RAX:  RBX: 
> > 0046 RCX: 0003
> > Jan  2 13:24:08 mail kernel: [3531872.484552] RDX:  RSI: 
> > 0086 RDI: 0300
> > Jan  2 13:24:08 mail kernel: [3531872.484554] RBP: 88011fd83da8 R08: 
> > 000a R09: 
> > Jan  2 13:24:08 mail kernel: [3531872.484556] R10:  R11: 
> >  R12: 0c00
> > Jan  2 13:24:08 mail kernel: [3531872.484558] R13: 0f00 R14: 
> > 88011fd8e700 R15: 
> > Jan  2 13:24:08 mail kernel: [3531872.484560] FS:  () 
> > GS:88011fd8() knlGS:
> > Jan  2 13:24:08 mail kernel: [3531872.484562] CS:  0010 DS:  ES:  
> > CR0: 8005003b
> > Jan  2 13:24:08 mail kernel: [3531872.484564] CR2: 7fc17c266000 CR3: 
> > 7a52d000 CR4: 06e0
> > Jan  2 13:24:08 mail kernel: [3531872.484570] DR0:  DR1: 
> >  DR2: 
> > Jan  2 13:24:08 mail kernel: [3531872.484575] DR3:  DR6: 
> > 0ff0 DR7: 0400
> > Jan  2 13:24:08 mail kernel: [3531872.484577] Process swapper/3 (pid: 0, 
> > threadinfo 880118b4e000, task 880118b516e0)
> > Jan  2 13:24:08 mail kernel: [3531872.484579] Stack:
> > Jan  2 13:24:08 mail kernel: [3531872.484610]   
> > 2710 81c30900 81c30a00
> > Jan  2 13:24:08 mail kernel: [3531872.484614]  88011fd83dc8 
> > 810323ca 000609111

Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Steven Rostedt
On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
> This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu SMP 
> Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)

3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
to file a Bugzilla (or whatever they call it) with them.

> 
> Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall on 
> CPU 3 (t=0 jiffies)

0 jiffies?

> Jan  2 13:24:08 mail kernel: [3531872.484398] sending NMI to all CPUs:
> Jan  2 13:24:08 mail kernel: [3531872.484430] NMI backtrace for cpu 3
> Jan  2 13:24:08 mail kernel: [3531872.484432] CPU 3
> Jan  2 13:24:08 mail kernel: [3531872.484433] Modules linked in: btrfs 
> zlib_deflate libcrc32c ufs vfat msdos fat xfs ip6_queue 
> nf_conntrack_proto_udplite xt_TCPMSS xt_sctp ip6t_LOG ip6t_REJECT 
> nf_conntrack_ipv6 ip6table_raw ip6table_mangle ip6table_filter act_police 
> cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb sch_hfsc 
> sch_ingress sch_sfq xt_time xt_connlimit xt_realm xt_addrtype iptable_raw 
> xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP 
> ipt_MASQUERADE ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah xt_set ip_set nf_nat_tftp 
> nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre 
> nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda 
> nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp 
> nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink 
> nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc 
> nf_conntrack_h323 nf_conntrack_ftp xt_TPROXY nf_tproxy_core ip6_tables 
> nf_defrag_i
>  pv6 xt_tcpmss xt_pkttype xt
> Jan  2 13:24:08 mail kernel: _physdev xt_owner xt_NFQUEUE xt_NFLOG 
> nfnetlink_log xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange 
> xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_connmark 
> xt_CLASSIFY xt_AUDIT ipt_LOG xt_tcpudp xt_state iptable_nat nf_nat 
> nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink 
> iptable_filter ip_tables x_tables ext2 psmouse serio_raw virtio_balloon lp 
> parport floppy
> Jan  2 13:24:08 mail kernel: [3531872.484533]
> Jan  2 13:24:08 mail kernel: [3531872.484536] Pid: 0, comm: swapper/3 Not 
> tainted 3.2.0-33-virtual #52-Ubuntu Bochs Bochs
> Jan  2 13:24:08 mail kernel: [3531872.484539] RIP: 0010:[]  
> [] flat_send_IPI_all+0xaf/0xd0
> Jan  2 13:24:08 mail kernel: [3531872.484548] RSP: 0018:88011fd83d88  
> EFLAGS: 00010006
> Jan  2 13:24:08 mail kernel: [3531872.484550] RAX:  RBX: 
> 0046 RCX: 0003
> Jan  2 13:24:08 mail kernel: [3531872.484552] RDX:  RSI: 
> 0086 RDI: 0300
> Jan  2 13:24:08 mail kernel: [3531872.484554] RBP: 88011fd83da8 R08: 
> 000a R09: 
> Jan  2 13:24:08 mail kernel: [3531872.484556] R10:  R11: 
>  R12: 0c00
> Jan  2 13:24:08 mail kernel: [3531872.484558] R13: 0f00 R14: 
> 88011fd8e700 R15: 
> Jan  2 13:24:08 mail kernel: [3531872.484560] FS:  () 
> GS:88011fd8() knlGS:
> Jan  2 13:24:08 mail kernel: [3531872.484562] CS:  0010 DS:  ES:  
> CR0: 8005003b
> Jan  2 13:24:08 mail kernel: [3531872.484564] CR2: 7fc17c266000 CR3: 
> 7a52d000 CR4: 06e0
> Jan  2 13:24:08 mail kernel: [3531872.484570] DR0:  DR1: 
>  DR2: 
> Jan  2 13:24:08 mail kernel: [3531872.484575] DR3:  DR6: 
> 0ff0 DR7: 0400
> Jan  2 13:24:08 mail kernel: [3531872.484577] Process swapper/3 (pid: 0, 
> threadinfo 880118b4e000, task 880118b516e0)
> Jan  2 13:24:08 mail kernel: [3531872.484579] Stack:
> Jan  2 13:24:08 mail kernel: [3531872.484610]   
> 2710 81c30900 81c30a00
> Jan  2 13:24:08 mail kernel: [3531872.484614]  88011fd83dc8 
> 810323ca 000609111e00 81c30900
> Jan  2 13:24:08 mail kernel: [3531872.484617]  88011fd83de8 
> 810ddf37 88011fd83e18 88011fd8ea60
> Jan  2 13:24:08 mail kernel: [3531872.484620] Call Trace:
> Jan  2 13:24:08 mail kernel: [3531872.484652]  
> Jan  2 13:24:08 mail kernel: [3531872.484689]  [] 
> arch_trigger_all_cpu_backtrace+0x5a/0x90
> Jan  2 13:24:08 mail kernel: [3531872.484703]  [] 
> check_cpu_stall.isra.36+0x97/0xf0
> Jan  2 13:24:08 mail kernel: [3531872.484706]  [] 
> __rcu_pending+0x38/0x1b0
> Jan  2 13:24:08 mail kernel: [3531872.484709]  [] 
> rcu_check_callbacks+0x1cb/0x1e0
> Jan  2 13:24:08 mail kernel: [3531

INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Ralf Hildebrandt
This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu SMP 
Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)

Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall on 
CPU 3 (t=0 jiffies)
Jan  2 13:24:08 mail kernel: [3531872.484398] sending NMI to all CPUs:
Jan  2 13:24:08 mail kernel: [3531872.484430] NMI backtrace for cpu 3
Jan  2 13:24:08 mail kernel: [3531872.484432] CPU 3
Jan  2 13:24:08 mail kernel: [3531872.484433] Modules linked in: btrfs 
zlib_deflate libcrc32c ufs vfat msdos fat xfs ip6_queue 
nf_conntrack_proto_udplite xt_TCPMSS xt_sctp ip6t_LOG ip6t_REJECT 
nf_conntrack_ipv6 ip6table_raw ip6table_mangle ip6table_filter act_police 
cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb sch_hfsc sch_ingress 
sch_sfq xt_time xt_connlimit xt_realm xt_addrtype iptable_raw xt_comment 
xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE 
ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah xt_set ip_set nf_nat_tftp 
nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre 
nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda 
nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp 
nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink 
nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc 
nf_conntrack_h323 nf_conntrack_ftp xt_TPROXY nf_tproxy_core ip6_tables 
nf_defrag_i
 pv6 xt_tcpmss xt_pkttype xt
Jan  2 13:24:08 mail kernel: _physdev xt_owner xt_NFQUEUE xt_NFLOG 
nfnetlink_log xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange 
xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_connmark 
xt_CLASSIFY xt_AUDIT ipt_LOG xt_tcpudp xt_state iptable_nat nf_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink 
iptable_filter ip_tables x_tables ext2 psmouse serio_raw virtio_balloon lp 
parport floppy
Jan  2 13:24:08 mail kernel: [3531872.484533]
Jan  2 13:24:08 mail kernel: [3531872.484536] Pid: 0, comm: swapper/3 Not 
tainted 3.2.0-33-virtual #52-Ubuntu Bochs Bochs
Jan  2 13:24:08 mail kernel: [3531872.484539] RIP: 0010:[]  
[] flat_send_IPI_all+0xaf/0xd0
Jan  2 13:24:08 mail kernel: [3531872.484548] RSP: 0018:88011fd83d88  
EFLAGS: 00010006
Jan  2 13:24:08 mail kernel: [3531872.484550] RAX:  RBX: 
0046 RCX: 0003
Jan  2 13:24:08 mail kernel: [3531872.484552] RDX:  RSI: 
0086 RDI: 0300
Jan  2 13:24:08 mail kernel: [3531872.484554] RBP: 88011fd83da8 R08: 
000a R09: 
Jan  2 13:24:08 mail kernel: [3531872.484556] R10:  R11: 
 R12: 0c00
Jan  2 13:24:08 mail kernel: [3531872.484558] R13: 0f00 R14: 
88011fd8e700 R15: 
Jan  2 13:24:08 mail kernel: [3531872.484560] FS:  () 
GS:88011fd8() knlGS:
Jan  2 13:24:08 mail kernel: [3531872.484562] CS:  0010 DS:  ES:  CR0: 
8005003b
Jan  2 13:24:08 mail kernel: [3531872.484564] CR2: 7fc17c266000 CR3: 
7a52d000 CR4: 06e0
Jan  2 13:24:08 mail kernel: [3531872.484570] DR0:  DR1: 
 DR2: 
Jan  2 13:24:08 mail kernel: [3531872.484575] DR3:  DR6: 
0ff0 DR7: 0400
Jan  2 13:24:08 mail kernel: [3531872.484577] Process swapper/3 (pid: 0, 
threadinfo 880118b4e000, task 880118b516e0)
Jan  2 13:24:08 mail kernel: [3531872.484579] Stack:
Jan  2 13:24:08 mail kernel: [3531872.484610]   
2710 81c30900 81c30a00
Jan  2 13:24:08 mail kernel: [3531872.484614]  88011fd83dc8 
810323ca 000609111e00 81c30900
Jan  2 13:24:08 mail kernel: [3531872.484617]  88011fd83de8 
810ddf37 88011fd83e18 88011fd8ea60
Jan  2 13:24:08 mail kernel: [3531872.484620] Call Trace:
Jan  2 13:24:08 mail kernel: [3531872.484652]  
Jan  2 13:24:08 mail kernel: [3531872.484689]  [] 
arch_trigger_all_cpu_backtrace+0x5a/0x90
Jan  2 13:24:08 mail kernel: [3531872.484703]  [] 
check_cpu_stall.isra.36+0x97/0xf0
Jan  2 13:24:08 mail kernel: [3531872.484706]  [] 
__rcu_pending+0x38/0x1b0
Jan  2 13:24:08 mail kernel: [3531872.484709]  [] 
rcu_check_callbacks+0x1cb/0x1e0
Jan  2 13:24:08 mail kernel: [3531872.484719]  [] 
update_process_times+0x48/0x90
Jan  2 13:24:08 mail kernel: [3531872.484725]  [] 
tick_sched_timer+0x64/0xc0
Jan  2 13:24:08 mail kernel: [3531872.484730]  [] 
__run_hrtimer+0x78/0x1f0
Jan  2 13:24:08 mail kernel: [3531872.484733]  [] ? 
tick_nohz_handler+0x100/0x100
Jan  2 13:24:08 mail kernel: [3531872.484736]  [] 
hrtimer_interrupt+0xf7/0x230
Jan  2 13:24:08 mail kernel: [3531872.484742]  [] 
smp_apic_timer_interrupt+0x69/0x99
Jan  2 13:24:08 mail kernel: [3531872.484745]  [] 
apic_timer_interrupt+0x6e/0x80
Jan  2 13:24:08 mail kernel: [3531872.484747]  
Jan  2 13:24:08

INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Ralf Hildebrandt
This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu SMP 
Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)

Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall on 
CPU 3 (t=0 jiffies)
Jan  2 13:24:08 mail kernel: [3531872.484398] sending NMI to all CPUs:
Jan  2 13:24:08 mail kernel: [3531872.484430] NMI backtrace for cpu 3
Jan  2 13:24:08 mail kernel: [3531872.484432] CPU 3
Jan  2 13:24:08 mail kernel: [3531872.484433] Modules linked in: btrfs 
zlib_deflate libcrc32c ufs vfat msdos fat xfs ip6_queue 
nf_conntrack_proto_udplite xt_TCPMSS xt_sctp ip6t_LOG ip6t_REJECT 
nf_conntrack_ipv6 ip6table_raw ip6table_mangle ip6table_filter act_police 
cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb sch_hfsc sch_ingress 
sch_sfq xt_time xt_connlimit xt_realm xt_addrtype iptable_raw xt_comment 
xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP ipt_MASQUERADE 
ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah xt_set ip_set nf_nat_tftp 
nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre 
nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda 
nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp 
nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink 
nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc 
nf_conntrack_h323 nf_conntrack_ftp xt_TPROXY nf_tproxy_core ip6_tables 
nf_defrag_i
 pv6 xt_tcpmss xt_pkttype xt
Jan  2 13:24:08 mail kernel: _physdev xt_owner xt_NFQUEUE xt_NFLOG 
nfnetlink_log xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange 
xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_connmark 
xt_CLASSIFY xt_AUDIT ipt_LOG xt_tcpudp xt_state iptable_nat nf_nat 
nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink 
iptable_filter ip_tables x_tables ext2 psmouse serio_raw virtio_balloon lp 
parport floppy
Jan  2 13:24:08 mail kernel: [3531872.484533]
Jan  2 13:24:08 mail kernel: [3531872.484536] Pid: 0, comm: swapper/3 Not 
tainted 3.2.0-33-virtual #52-Ubuntu Bochs Bochs
Jan  2 13:24:08 mail kernel: [3531872.484539] RIP: 0010:[81036d5f]  
[81036d5f] flat_send_IPI_all+0xaf/0xd0
Jan  2 13:24:08 mail kernel: [3531872.484548] RSP: 0018:88011fd83d88  
EFLAGS: 00010006
Jan  2 13:24:08 mail kernel: [3531872.484550] RAX:  RBX: 
0046 RCX: 0003
Jan  2 13:24:08 mail kernel: [3531872.484552] RDX:  RSI: 
0086 RDI: 0300
Jan  2 13:24:08 mail kernel: [3531872.484554] RBP: 88011fd83da8 R08: 
000a R09: 
Jan  2 13:24:08 mail kernel: [3531872.484556] R10:  R11: 
 R12: 0c00
Jan  2 13:24:08 mail kernel: [3531872.484558] R13: 0f00 R14: 
88011fd8e700 R15: 
Jan  2 13:24:08 mail kernel: [3531872.484560] FS:  () 
GS:88011fd8() knlGS:
Jan  2 13:24:08 mail kernel: [3531872.484562] CS:  0010 DS:  ES:  CR0: 
8005003b
Jan  2 13:24:08 mail kernel: [3531872.484564] CR2: 7fc17c266000 CR3: 
7a52d000 CR4: 06e0
Jan  2 13:24:08 mail kernel: [3531872.484570] DR0:  DR1: 
 DR2: 
Jan  2 13:24:08 mail kernel: [3531872.484575] DR3:  DR6: 
0ff0 DR7: 0400
Jan  2 13:24:08 mail kernel: [3531872.484577] Process swapper/3 (pid: 0, 
threadinfo 880118b4e000, task 880118b516e0)
Jan  2 13:24:08 mail kernel: [3531872.484579] Stack:
Jan  2 13:24:08 mail kernel: [3531872.484610]   
2710 81c30900 81c30a00
Jan  2 13:24:08 mail kernel: [3531872.484614]  88011fd83dc8 
810323ca 000609111e00 81c30900
Jan  2 13:24:08 mail kernel: [3531872.484617]  88011fd83de8 
810ddf37 88011fd83e18 88011fd8ea60
Jan  2 13:24:08 mail kernel: [3531872.484620] Call Trace:
Jan  2 13:24:08 mail kernel: [3531872.484652]  IRQ
Jan  2 13:24:08 mail kernel: [3531872.484689]  [810323ca] 
arch_trigger_all_cpu_backtrace+0x5a/0x90
Jan  2 13:24:08 mail kernel: [3531872.484703]  [810ddf37] 
check_cpu_stall.isra.36+0x97/0xf0
Jan  2 13:24:08 mail kernel: [3531872.484706]  [810ddfc8] 
__rcu_pending+0x38/0x1b0
Jan  2 13:24:08 mail kernel: [3531872.484709]  [810de58b] 
rcu_check_callbacks+0x1cb/0x1e0
Jan  2 13:24:08 mail kernel: [3531872.484719]  [81076fb8] 
update_process_times+0x48/0x90
Jan  2 13:24:08 mail kernel: [3531872.484725]  [8109a5f4] 
tick_sched_timer+0x64/0xc0
Jan  2 13:24:08 mail kernel: [3531872.484730]  [8108cd78] 
__run_hrtimer+0x78/0x1f0
Jan  2 13:24:08 mail kernel: [3531872.484733]  [8109a590] ? 
tick_nohz_handler+0x100/0x100
Jan  2 13:24:08 mail kernel: [3531872.484736]  [8108d6a7] 
hrtimer_interrupt+0xf7/0x230
Jan  2 13:24:08 mail kernel: [3531872.484742

Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Steven Rostedt
On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
 This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu SMP 
 Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)

3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
to file a Bugzilla (or whatever they call it) with them.

 
 Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall on 
 CPU 3 (t=0 jiffies)

0 jiffies?

 Jan  2 13:24:08 mail kernel: [3531872.484398] sending NMI to all CPUs:
 Jan  2 13:24:08 mail kernel: [3531872.484430] NMI backtrace for cpu 3
 Jan  2 13:24:08 mail kernel: [3531872.484432] CPU 3
 Jan  2 13:24:08 mail kernel: [3531872.484433] Modules linked in: btrfs 
 zlib_deflate libcrc32c ufs vfat msdos fat xfs ip6_queue 
 nf_conntrack_proto_udplite xt_TCPMSS xt_sctp ip6t_LOG ip6t_REJECT 
 nf_conntrack_ipv6 ip6table_raw ip6table_mangle ip6table_filter act_police 
 cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb sch_hfsc 
 sch_ingress sch_sfq xt_time xt_connlimit xt_realm xt_addrtype iptable_raw 
 xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP 
 ipt_MASQUERADE ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah xt_set ip_set nf_nat_tftp 
 nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp nf_nat_proto_gre 
 nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp nf_conntrack_amanda 
 nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip nf_conntrack_proto_sctp 
 nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_netlink 
 nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_irc 
 nf_conntrack_h323 nf_conntrack_ftp xt_TPROXY nf_tproxy_core ip6_tables 
 nf_defrag_i
  pv6 xt_tcpmss xt_pkttype xt
 Jan  2 13:24:08 mail kernel: _physdev xt_owner xt_NFQUEUE xt_NFLOG 
 nfnetlink_log xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange 
 xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_connmark 
 xt_CLASSIFY xt_AUDIT ipt_LOG xt_tcpudp xt_state iptable_nat nf_nat 
 nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink 
 iptable_filter ip_tables x_tables ext2 psmouse serio_raw virtio_balloon lp 
 parport floppy
 Jan  2 13:24:08 mail kernel: [3531872.484533]
 Jan  2 13:24:08 mail kernel: [3531872.484536] Pid: 0, comm: swapper/3 Not 
 tainted 3.2.0-33-virtual #52-Ubuntu Bochs Bochs
 Jan  2 13:24:08 mail kernel: [3531872.484539] RIP: 0010:[81036d5f]  
 [81036d5f] flat_send_IPI_all+0xaf/0xd0
 Jan  2 13:24:08 mail kernel: [3531872.484548] RSP: 0018:88011fd83d88  
 EFLAGS: 00010006
 Jan  2 13:24:08 mail kernel: [3531872.484550] RAX:  RBX: 
 0046 RCX: 0003
 Jan  2 13:24:08 mail kernel: [3531872.484552] RDX:  RSI: 
 0086 RDI: 0300
 Jan  2 13:24:08 mail kernel: [3531872.484554] RBP: 88011fd83da8 R08: 
 000a R09: 
 Jan  2 13:24:08 mail kernel: [3531872.484556] R10:  R11: 
  R12: 0c00
 Jan  2 13:24:08 mail kernel: [3531872.484558] R13: 0f00 R14: 
 88011fd8e700 R15: 
 Jan  2 13:24:08 mail kernel: [3531872.484560] FS:  () 
 GS:88011fd8() knlGS:
 Jan  2 13:24:08 mail kernel: [3531872.484562] CS:  0010 DS:  ES:  
 CR0: 8005003b
 Jan  2 13:24:08 mail kernel: [3531872.484564] CR2: 7fc17c266000 CR3: 
 7a52d000 CR4: 06e0
 Jan  2 13:24:08 mail kernel: [3531872.484570] DR0:  DR1: 
  DR2: 
 Jan  2 13:24:08 mail kernel: [3531872.484575] DR3:  DR6: 
 0ff0 DR7: 0400
 Jan  2 13:24:08 mail kernel: [3531872.484577] Process swapper/3 (pid: 0, 
 threadinfo 880118b4e000, task 880118b516e0)
 Jan  2 13:24:08 mail kernel: [3531872.484579] Stack:
 Jan  2 13:24:08 mail kernel: [3531872.484610]   
 2710 81c30900 81c30a00
 Jan  2 13:24:08 mail kernel: [3531872.484614]  88011fd83dc8 
 810323ca 000609111e00 81c30900
 Jan  2 13:24:08 mail kernel: [3531872.484617]  88011fd83de8 
 810ddf37 88011fd83e18 88011fd8ea60
 Jan  2 13:24:08 mail kernel: [3531872.484620] Call Trace:
 Jan  2 13:24:08 mail kernel: [3531872.484652]  IRQ
 Jan  2 13:24:08 mail kernel: [3531872.484689]  [810323ca] 
 arch_trigger_all_cpu_backtrace+0x5a/0x90
 Jan  2 13:24:08 mail kernel: [3531872.484703]  [810ddf37] 
 check_cpu_stall.isra.36+0x97/0xf0
 Jan  2 13:24:08 mail kernel: [3531872.484706]  [810ddfc8] 
 __rcu_pending+0x38/0x1b0
 Jan  2 13:24:08 mail kernel: [3531872.484709]  [810de58b] 
 rcu_check_callbacks+0x1cb/0x1e0
 Jan  2 13:24:08 mail kernel: [3531872.484719]  [81076fb8] 
 update_process_times+0x48/0x90
 Jan  2 13:24:08 mail kernel: [3531872.484725]  [8109a5f4] 
 tick_sched_timer+0x64/0xc0
 Jan  2 13:24:08 mail kernel: [3531872.484730

Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Paul E. McKenney
On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
 On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
  This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu 
  SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)
 
 3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
 to file a Bugzilla (or whatever they call it) with them.
 
  
  Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall 
  on CPU 3 (t=0 jiffies)
 
 0 jiffies?

One possibility is that they are missing commit a10d206e (rcu: Fix
day-one dyntick-idle stall-warning bug), at least assuming that
CONFIG_PREEMPT=n.  This went into 3.6, IIRC.

Thanx, Paul

  Jan  2 13:24:08 mail kernel: [3531872.484398] sending NMI to all CPUs:
  Jan  2 13:24:08 mail kernel: [3531872.484430] NMI backtrace for cpu 3
  Jan  2 13:24:08 mail kernel: [3531872.484432] CPU 3
  Jan  2 13:24:08 mail kernel: [3531872.484433] Modules linked in: btrfs 
  zlib_deflate libcrc32c ufs vfat msdos fat xfs ip6_queue 
  nf_conntrack_proto_udplite xt_TCPMSS xt_sctp ip6t_LOG ip6t_REJECT 
  nf_conntrack_ipv6 ip6table_raw ip6table_mangle ip6table_filter act_police 
  cls_basic cls_flow cls_fw cls_u32 sch_tbf sch_prio sch_htb sch_hfsc 
  sch_ingress sch_sfq xt_time xt_connlimit xt_realm xt_addrtype iptable_raw 
  xt_comment xt_recent xt_policy ipt_ULOG ipt_REJECT ipt_REDIRECT ipt_NETMAP 
  ipt_MASQUERADE ipt_ECN ipt_ecn ipt_CLUSTERIP ipt_ah xt_set ip_set 
  nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_nat_sip nf_nat_pptp 
  nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda ts_kmp 
  nf_conntrack_amanda nf_conntrack_sane nf_conntrack_tftp nf_conntrack_sip 
  nf_conntrack_proto_sctp nf_conntrack_pptp nf_conntrack_proto_gre 
  nf_conntrack_netlink nf_conntrack_netbios_ns nf_conntrack_broadcast 
  nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp xt_TPROXY 
  nf_tproxy_core ip6_tables nf_defrag_i
   pv6 xt_tcpmss xt_pkttype xt
  Jan  2 13:24:08 mail kernel: _physdev xt_owner xt_NFQUEUE xt_NFLOG 
  nfnetlink_log xt_multiport xt_mark xt_mac xt_limit xt_length xt_iprange 
  xt_helper xt_hashlimit xt_DSCP xt_dscp xt_dccp xt_conntrack xt_connmark 
  xt_CLASSIFY xt_AUDIT ipt_LOG xt_tcpudp xt_state iptable_nat nf_nat 
  nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack iptable_mangle nfnetlink 
  iptable_filter ip_tables x_tables ext2 psmouse serio_raw virtio_balloon lp 
  parport floppy
  Jan  2 13:24:08 mail kernel: [3531872.484533]
  Jan  2 13:24:08 mail kernel: [3531872.484536] Pid: 0, comm: swapper/3 Not 
  tainted 3.2.0-33-virtual #52-Ubuntu Bochs Bochs
  Jan  2 13:24:08 mail kernel: [3531872.484539] RIP: 
  0010:[81036d5f]  [81036d5f] flat_send_IPI_all+0xaf/0xd0
  Jan  2 13:24:08 mail kernel: [3531872.484548] RSP: 0018:88011fd83d88  
  EFLAGS: 00010006
  Jan  2 13:24:08 mail kernel: [3531872.484550] RAX:  RBX: 
  0046 RCX: 0003
  Jan  2 13:24:08 mail kernel: [3531872.484552] RDX:  RSI: 
  0086 RDI: 0300
  Jan  2 13:24:08 mail kernel: [3531872.484554] RBP: 88011fd83da8 R08: 
  000a R09: 
  Jan  2 13:24:08 mail kernel: [3531872.484556] R10:  R11: 
   R12: 0c00
  Jan  2 13:24:08 mail kernel: [3531872.484558] R13: 0f00 R14: 
  88011fd8e700 R15: 
  Jan  2 13:24:08 mail kernel: [3531872.484560] FS:  () 
  GS:88011fd8() knlGS:
  Jan  2 13:24:08 mail kernel: [3531872.484562] CS:  0010 DS:  ES:  
  CR0: 8005003b
  Jan  2 13:24:08 mail kernel: [3531872.484564] CR2: 7fc17c266000 CR3: 
  7a52d000 CR4: 06e0
  Jan  2 13:24:08 mail kernel: [3531872.484570] DR0:  DR1: 
   DR2: 
  Jan  2 13:24:08 mail kernel: [3531872.484575] DR3:  DR6: 
  0ff0 DR7: 0400
  Jan  2 13:24:08 mail kernel: [3531872.484577] Process swapper/3 (pid: 0, 
  threadinfo 880118b4e000, task 880118b516e0)
  Jan  2 13:24:08 mail kernel: [3531872.484579] Stack:
  Jan  2 13:24:08 mail kernel: [3531872.484610]   
  2710 81c30900 81c30a00
  Jan  2 13:24:08 mail kernel: [3531872.484614]  88011fd83dc8 
  810323ca 000609111e00 81c30900
  Jan  2 13:24:08 mail kernel: [3531872.484617]  88011fd83de8 
  810ddf37 88011fd83e18 88011fd8ea60
  Jan  2 13:24:08 mail kernel: [3531872.484620] Call Trace:
  Jan  2 13:24:08 mail kernel: [3531872.484652]  IRQ
  Jan  2 13:24:08 mail kernel: [3531872.484689]  [810323ca] 
  arch_trigger_all_cpu_backtrace+0x5a/0x90
  Jan  2 13:24:08 mail kernel: [3531872.484703]  [810ddf37] 
  check_cpu_stall.isra.36+0x97/0xf0
  Jan  2 13:24:08 mail kernel: [3531872.484706

Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Steven Rostedt
On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
 On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
  On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
   This happened of a virtual guest (Linux mail 3.2.0-35-virtual #55-Ubuntu 
   SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux)
  
  3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
  to file a Bugzilla (or whatever they call it) with them.
  
   
   Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected stall 
   on CPU 3 (t=0 jiffies)
  
  0 jiffies?
 
 One possibility is that they are missing commit a10d206e (rcu: Fix
 day-one dyntick-idle stall-warning bug), at least assuming that
 CONFIG_PREEMPT=n.  This went into 3.6, IIRC.

If that does fix the issue, we probably should send that off to Ben for
inclusion into 3.2 stable.

Thanks,

-- Steve

 
   Thanx, Paul
 



--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


Re: INFO: rcu_bh detected stall on CPU 3 (t=0 jiffies)

2013-01-02 Thread Paul E. McKenney
On Wed, Jan 02, 2013 at 07:49:23PM -0500, Steven Rostedt wrote:
 On Wed, 2013-01-02 at 13:29 -0800, Paul E. McKenney wrote:
  On Wed, Jan 02, 2013 at 03:56:30PM -0500, Steven Rostedt wrote:
   On Wed, Jan 02, 2013 at 08:09:20PM +0100, Ralf Hildebrandt wrote:
This happened of a virtual guest (Linux mail 3.2.0-35-virtual 
#55-Ubuntu SMP Wed Dec 5 18:02:05 UTC 2012 x86_64 x86_64 x86_64 
GNU/Linux)
   
   3.2 is rather old and this looks to be a stock Ubuntu kernel. May want
   to file a Bugzilla (or whatever they call it) with them.
   

Jan  2 13:24:08 mail kernel: [3531872.484283] INFO: rcu_bh detected 
stall on CPU 3 (t=0 jiffies)
   
   0 jiffies?
  
  One possibility is that they are missing commit a10d206e (rcu: Fix
  day-one dyntick-idle stall-warning bug), at least assuming that
  CONFIG_PREEMPT=n.  This went into 3.6, IIRC.
 
 If that does fix the issue, we probably should send that off to Ben for
 inclusion into 3.2 stable.

Looks like you already CCed him, thank you!  ;-)

Thanx, Paul

 Thanks,
 
 -- Steve
 
  
  Thanx, Paul
  
 
 
 
 --
 To unsubscribe from this list: send the line unsubscribe linux-kernel in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
 Please read the FAQ at  http://www.tux.org/lkml/
 

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/