[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From mainam...@in.ibm.com 2019-03-15 02:15 EDT---
*** Bug 168118 has been marked as a duplicate of this bug. ***
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1773964
Title:
IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls
on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n
Status in The Ubuntu-power-systems project:
Fix Released
Status in linux package in Ubuntu:
Fix Released
Status in linux source package in Bionic:
Fix Released
Bug description:
== SRU Justification ==
IBM is requesting these two commits in powerpc to fix an error they are
seeing:
IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on
CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n
The first commit reduces some existing high latency delays. The second
commit fixes the fact that the OPAL RTC driver does not sleep in case it gets
OPAL_BUSY or
OPAL_BUSY_EVENT from firmware.
== Fixes ==
34dd25de9fe3 ("powerpc/powernv: define a standard delay for OPAL_BUSY type
retry loops")
682e6b4da5cb ("rtc: opal: Fix OPAL RTC driver OPAL_BUSY loops")
== Regression Potential ==
Low. The first commit is specific to powerpc. The second commit is, commit
628daa8d5abf
and it been cc'd to upstream stable to fix a regression, so it has had
additional upstream review.
== Test Case ==
A test kernel was built with these patches and tested by the original bug
reporter.
The bug reporter states the test kernel resolved the bug.
== Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 ==
== Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 ==
State: Open by: swanman on 22 May 2018 09:17:33
Both w34 and wsbmc016 are stopped on this failure, where we get hung
up on the following FFDC gathering step in the OS:
#(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command
ppc64_cpu --frequency ignore_err=1
#(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency
, and the last console output shows:
# tail -200 /var/log/obmc-console.log
s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989
[13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881)
[13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks:
[13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=242996
[13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822)
[13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks:
[13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=243942
[13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916)
[13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks:
[13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=244940
[13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646)
[13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks:
[13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=245874
[13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909)
[13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks:
[13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=246892
[13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048)
[14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks:
[14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=247857
[14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185)
[14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks:
[14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=248819
[14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190)
[14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks:
[14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=249833
[14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288,
q=822138)
[14244.968067]I F: Ne talls onCPUs/tasks
[14244669838] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies,
g=30289, c=30288, q=825384)
[14308.748680] Id detected stalls on CPUs/tasks:
[14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=251755
[14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445)
[14333.081775] Watchdog CPU:132 Hard LOCKUP
[14344.750295] Watchdog CPU:132 became unstuck
[14372.735558] INFO: rcu_sched detected stalls on
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From gwal...@br.ibm.com 2018-06-15 08:07 EDT---
I'm going to be on vacation next weeks, let the kernel backports team know if
you all need any kind of support.
https://github.ibm.com/gwalbon/kernel-backports/wiki/Bugzilla
-::-Request-Backports
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1773964
Title:
IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls
on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n
Status in The Ubuntu-power-systems project:
In Progress
Status in linux package in Ubuntu:
In Progress
Status in linux source package in Bionic:
In Progress
Bug description:
== SRU Justification ==
IBM is requesting these two commits in powerpc to fix an error they are
seeing:
IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on
CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n
The first commit reduces some existing high latency delays. The second
commit fixes the fact that the OPAL RTC driver does not sleep in case it gets
OPAL_BUSY or
OPAL_BUSY_EVENT from firmware.
== Fixes ==
34dd25de9fe3 ("powerpc/powernv: define a standard delay for OPAL_BUSY type
retry loops")
682e6b4da5cb ("rtc: opal: Fix OPAL RTC driver OPAL_BUSY loops")
== Regression Potential ==
Low. The first commit is specific to powerpc. The second commit is, commit
628daa8d5abf
and it been cc'd to upstream stable to fix a regression, so it has had
additional upstream review.
== Test Case ==
A test kernel was built with these patches and tested by the original bug
reporter.
The bug reporter states the test kernel resolved the bug.
== Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 ==
== Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 ==
State: Open by: swanman on 22 May 2018 09:17:33
Both w34 and wsbmc016 are stopped on this failure, where we get hung
up on the following FFDC gathering step in the OS:
#(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command
ppc64_cpu --frequency ignore_err=1
#(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency
, and the last console output shows:
# tail -200 /var/log/obmc-console.log
s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989
[13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881)
[13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks:
[13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=242996
[13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822)
[13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks:
[13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=243942
[13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916)
[13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks:
[13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=244940
[13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646)
[13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks:
[13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=245874
[13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909)
[13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks:
[13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=246892
[13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048)
[14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks:
[14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=247857
[14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185)
[14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks:
[14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=248819
[14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190)
[14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks:
[14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=249833
[14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288,
q=822138)
[14244.968067]I F: Ne talls onCPUs/tasks
[14244669838] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies,
g=30289, c=30288, q=825384)
[14308.748680] Id detected stalls on CPUs/tasks:
[14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0
softirq=2689/2689 fqs=251755
[14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445)
[14333.081775] Watchdog
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-06-12 10:46 EDT--- State: Working by: yadlapat on 12 June 2018 09:43:39 #=#=# 2018-06-12 09:43:37 (CDT) #=#=# New Priority = [2] #=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=#=# -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=255493 [14564.664470] (detected by 3, t=3299596
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From shigb...@in.ibm.com 2018-06-11 11:24 EDT--- Stephanie Swanson has confirmed to have tested with the new kernel and the error was not reproduced. She has also mentioned that the original bug was occurring intermittently and there has been no recreates of this issue with the new kernel. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: Incomplete Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP)
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-06-05 14:35 EDT--- State: Working by: yadlapat on 05 June 2018 13:32:15 Opened https://w3.rchland.ibm.com/projects/bestquest/?defect=SW432166 to track if this patch is needed for Pegas -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=255493 [14564.664470]
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From bren...@br.ibm.com 2018-06-05 14:15 EDT--- hi David, > The failure was intermittent. Automated tests are running on the system. > Time will tell if this resolves the issue. Any update on the failures? -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=255493 [14564.664470] (detected by 3,
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-05-30 19:55 EDT--- State: Working by: dlshaw on 30 May 2018 18:45:42 #=#=#=# David L. Shaw (dls...@us.ibm.com) - 2018-05-30 18:45:40 (CDT) - POWER Firmware #=#=#=# The failure was intermittent. Automated tests are running on the system. Time will tell if this resolves the issue. #=#=#=# end of update #=#=#=# -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks:
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-05-30 17:35 EDT--- State: Working by: dlshaw on 30 May 2018 16:24:44 #=#=#=# David L. Shaw (dls...@us.ibm.com) - 2018-05-30 16:24:41 (CDT) - POWER Firmware #=#=#=# I think I installed this test kernel correctly on w34's OS (w34L.aus.stglabs.ibm.com). Let me know if I did something wrong and need to fix it. Before installing this test kernel it had Ubuntu 18.04 (Bionic) kernel: 4.15.0-22-generic Ran the following to install test kernel: root@w34L:~/SW430227# ls -lt total 48220 -rw-r--r-- 1 root root 31438592 May 29 11:14 linux-modules-extra-4.15.0-23-generic_4.15.0-23.26~lp1773964_ppc64el.deb -rw-r--r-- 1 root root 6152468 May 29 11:14 linux-image-unsigned-4.15.0-23-generic_4.15.0-23.26~lp1773964_ppc64el.deb -rw-r--r-- 1 root root 11767420 May 29 11:14 linux-modules-4.15.0-23-generic_4.15.0-23.26~lp1773964_ppc64el.deb root@w34L:~/SW430227# dpkg -i *.deb Selecting previously unselected package linux-image-unsigned-4.15.0-23-generic. (Reading database ... 175537 files and directories currently installed.) Preparing to unpack linux-image-unsigned-4.15.0-23-generic_4.15.0-23.26~lp1773964_ppc64el.deb ... Unpacking linux-image-unsigned-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... Selecting previously unselected package linux-modules-4.15.0-23-generic. Preparing to unpack linux-modules-4.15.0-23-generic_4.15.0-23.26~lp1773964_ppc64el.deb ... Unpacking linux-modules-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... Selecting previously unselected package linux-modules-extra-4.15.0-23-generic. Preparing to unpack linux-modules-extra-4.15.0-23-generic_4.15.0-23.26~lp1773964_ppc64el.deb ... Unpacking linux-modules-extra-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... Setting up linux-modules-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... Setting up linux-image-unsigned-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... I: /boot/vmlinux.old is now a symlink to vmlinux-4.15.0-22-generic I: /boot/initrd.img.old is now a symlink to initrd.img-4.15.0-22-generic I: /boot/vmlinux is now a symlink to vmlinux-4.15.0-23-generic I: /boot/initrd.img is now a symlink to initrd.img-4.15.0-23-generic Setting up linux-modules-extra-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... Processing triggers for linux-image-unsigned-4.15.0-23-generic (4.15.0-23.26~lp1773964) ... /etc/kernel/postinst.d/initramfs-tools: update-initramfs: Generating /boot/initrd.img-4.15.0-23-generic W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast /etc/kernel/postinst.d/zz-update-grub: Generating grub configuration file ... Found linux image: /boot/vmlinux-4.15.0-23-generic Found initrd image: /boot/initrd.img-4.15.0-23-generic Found linux image: /boot/vmlinux-4.15.0-22-generic Found initrd image: /boot/initrd.img-4.15.0-22-generic Found linux image: /boot/vmlinux-4.15.0-20-generic Found initrd image: /boot/initrd.img-4.15.0-20-generic Found linux image: /boot/vmlinux-4.15.0-15-generic Found initrd image: /boot/initrd.img-4.15.0-15-generic Found linux image: /boot/vmlinux-4.15.0-13-generic Found initrd image: /boot/initrd.img-4.15.0-13-generic done Rebooted the system and the new kernel is: Linux w34L 4.15.0-23-generic #26~lp1773964 SMP Tue May 29 16:03:09 UTC 2018 ppc64le ppc64le ppc64le GNU/Linux #=#=#=# end of update #=#=#=# -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-05-29 12:22 EDT--- State: Working by: cde00 on 29 May 2018 11:22:32 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: In Progress Status in linux package in Ubuntu: In Progress Status in linux source package in Bionic: In Progress Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=255493 [14564.664470] (detected by 3, t=3299596 jiffies, g=30289, c=30288, q=840539) [14628.665755] INFO: rcu_sched detected stalls on
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-05-29 11:05 EDT--- == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 State: Working by: cde00 on 29 May 2018 09:57:15 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: New Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=255493 [14564.664470] (detected by 3, t=3299596 jiffies, g=30289, c=30288, q=840539)
[Kernel-packages] [Bug 1773964] Comment bridged from LTC Bugzilla
--- Comment From cdead...@us.ibm.com 2018-05-29 10:55 EDT--- == Comment: #3 - Application Cdeadmin <> - 2018-05-22 12:42:02 == State: Open by: swanman on 22 May 2018 09:17:33 -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1773964 Title: IPL: ppc64_cpu --frequency hang with INFO: rcu_sched detected stalls on CPUs/tasks on w34 and wsbmc016 with 920.1714.20170330n Status in The Ubuntu-power-systems project: Triaged Status in linux package in Ubuntu: New Bug description: == Comment: #0 - Application Cdeadmin <> - 2018-05-22 09:36:12 == == Comment: #1 - Application Cdeadmin <> - 2018-05-22 09:36:14 == State: Open by: swanman on 22 May 2018 09:17:33 Both w34 and wsbmc016 are stopped on this failure, where we get hung up on the following FFDC gathering step in the OS: #(CDT) 2018/05/22 05:07:19.398218 - 0.211662 - Issuing: OS Execute Command ppc64_cpu --frequency ignore_err=1 #(CDT) 2018/05/22 05:07:19.437808 - 0.024108 - Issuing: ppc64_cpu --frequency , and the last console output shows: # tail -200 /var/log/obmc-console.log s this GP) idle=82a/140/0 softirq=2689/2689 fqs=241989 [13668.719850] (detected by 3, t=3075620 jiffies, g=30289, c=30288, q=796881) [13732.630691] INFO: rcu_sched detected stalls on CPUs/tasks: [13732.630727] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=242996 [13732.630757] (detected by 3, t=3091597 jiffies, g=30289, c=30288, q=799822) [13795.649568] INFO: rcu_sched detected stalls on CPUs/tasks: [13795.649597] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=243942 [13795.649631] (detected by 7, t=3107351 jiffies, g=30289, c=30288, q=802916) [13860.140483] INFO: rcu_sched detected stalls on CPUs/tasks: [13860.140516] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=244940 [13860.140557] (detected by 3, t=3123473 jiffies, g=30289, c=30288, q=806646) [13924.731404] INFO: rcu_sched detected stalls on CPUs/tasks: [13924.731438] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=245874 [13924.731484] (detected by 3, t=3139620 jiffies, g=30289, c=30288, q=809909) [13988.642290] INFO: rcu_sched detected stalls on CPUs/tasks: [13988.642321] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=246892 [13988.642362] (detected by 3, t=3155596 jiffies, g=30289, c=30288, q=813048) [14052.661176] INFO: rcu_sched detected stalls on CPUs/tasks: [14052.661211] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=247857 [14052.661266] (detected by 3, t=3171600 jiffies, g=30289, c=30288, q=816185) [14115.680024] INFO: rcu_sched detected stalls on CPUs/tasks: [14115.680051] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=248819 [14115.680085] (detected by 4, t=3187355 jiffies, g=30289, c=30288, q=819190) [14180.154911] INFO: rcu_sched detected stalls on CPUs/tasks: [14180.154945] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=249833 [14180.154986] (detected by 132, t=3203473 jiffies, g=30289, c=30288, q=822138) [14244.968067]I F: Ne talls onCPUs/tasks [14244669838] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=2[14244.669944] (detected by 3, t=3219600 jiffies, g=30289, c=30288, q=825384) [14308.748680] Id detected stalls on CPUs/tasks: [14308.748719] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=251755 [14308.748755] (detected by 3, t=3235620 jiffies, g=30289, c=30288, q=828445) [14333.081775] Watchdog CPU:132 Hard LOCKUP [14344.750295] Watchdog CPU:132 became unstuck [14372.735558] INFO: rcu_sched detected stalls on CPUs/tasks: [14372.735587] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=252722 [14372.735611] (detected by 6, t=3251616 jiffies, g=30289, c=30288, q=831375) [14435.910193] INFO: rcu_sched detected stalls on CPUs/tasks: [14435.910219] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=253675 [14435.910253] (detected by 3, t=3267409 jiffies, g=30289, c=30288, q=834375) [14500.167195] INFO: rcu_sched detected stalls on CPUs/tasks: [14500.167230] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=254601 [14500.167273] (detected by 134, t=3283473 jiffies, g=30289, c=30288, q=837600) [14564.664395] INFO: rcu_sched detected stalls on CPUs/tasks: [14564.664427] 75-: (5904 ticks this GP) idle=82a/140/0 softirq=2689/2689 fqs=255493 [14564.664470] (detected by 3, t=3299596 jiffies, g=30289, c=30288, q=840539) [14628.665755] INFO: rcu_sched detected stalls on CPUs/tasks:
