Public bug reported:
It was brought to my attention the following situation:
"""
At Nov 12 02:15:39 Juju node reports kernel soft lock up.
Nov 12 02:15:39 l1-bootjujuvm-1a-de kernel: [6323788.024017] BUG: soft lockup -
CPU#0 stuck for 22s! [khungtaskd:35]
Nov 12 02:15:39 l1-bootjujuvm-1a-de kernel: [6323840.040003] BUG: soft lockup -
CPU#1 stuck for 22s! [mongod:1575]
( jujuvm-var-log-2014-11-18-case-xxxxxxxxx.tar.bz2 )
machine-0: 2014-11-12 02:15:39 ERROR juju.state.apiserver.common
resource.go:102 error stopping *apiserver.pingTimeout resource: ping
timeout ( all-machines-juju-00075278-2014-11-18.log.bz2 )
juju failed and then restarted, causing openstack components to restart.
"""
After digging a bit I found the following stack traces:
"""
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.770725] INFO: task
jbd2/sda1-8:322 blocked for more than 120 seconds.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.785348] Not tainted
3.13.0-32-generic #57-Ubuntu
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.801256] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838388] jbd2/sda1-8 D
ffff88103f2d4440 0 322 2 0x00000000
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838396] ffff881022a3bbc8
0000000000000002 ffff88102296dfc0 ffff881022a3bfd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838403] 0000000000014440
0000000000014440 ffff88102296dfc0 ffff88103f2d4cd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838408] ffff88107ffba550
0000000000000002 ffffffff811ee000 ffff881022a3bc40
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838423] Call Trace:
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838438] [<ffffffff811ee000>] ?
generic_block_bmap+0x50/0x50
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838447] [<ffffffff817203fd>]
io_schedule+0x9d/0x140
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838468] [<ffffffff811ee00e>]
sleep_on_buffer+0xe/0x20
...
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838518] INFO: task
qemu-system-x86:26225 blocked for more than 120 seconds.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.884250] Not tainted
3.13.0-32-generic #57-Ubuntu
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.910724] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968688] qemu-system-x86 D
ffff88103f514440 0 26225 1 0x00000000
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968692] ffff8806707cbd28
0000000000000002 ffff88100e63dfc0 ffff8806707cbfd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968694] 0000000000014440
0000000000014440 ffff88100e63dfc0 ffff88103f514cd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968696] ffff88107ffba0e8
0000000000000002 ffffffff8114e190 ffff8806707cbda0
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968699] Call Trace:
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968704] [<ffffffff8114e190>] ?
wait_on_page_read+0x60/0x60
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968707] [<ffffffff817203fd>]
io_schedule+0x9d/0x140
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968715] [<ffffffff8114e19e>]
sleep_on_page+0xe/0x20
"""
** Affects: linux (Ubuntu)
Importance: Undecided
Assignee: Rafael David Tinoco (inaddy)
Status: Invalid
** Changed in: linux (Ubuntu)
Status: New => In Progress
** Changed in: linux (Ubuntu)
Assignee: (unassigned) => Rafael David Tinoco (inaddy)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1401181
Title:
Kernel Soft Lockup - CPU stuck for XXs!
Status in linux package in Ubuntu:
Invalid
Bug description:
It was brought to my attention the following situation:
"""
At Nov 12 02:15:39 Juju node reports kernel soft lock up.
Nov 12 02:15:39 l1-bootjujuvm-1a-de kernel: [6323788.024017] BUG: soft lockup
- CPU#0 stuck for 22s! [khungtaskd:35]
Nov 12 02:15:39 l1-bootjujuvm-1a-de kernel: [6323840.040003] BUG: soft lockup
- CPU#1 stuck for 22s! [mongod:1575]
( jujuvm-var-log-2014-11-18-case-xxxxxxxxx.tar.bz2 )
machine-0: 2014-11-12 02:15:39 ERROR juju.state.apiserver.common
resource.go:102 error stopping *apiserver.pingTimeout resource: ping
timeout ( all-machines-juju-00075278-2014-11-18.log.bz2 )
juju failed and then restarted, causing openstack components to restart.
"""
After digging a bit I found the following stack traces:
"""
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.770725] INFO: task
jbd2/sda1-8:322 blocked for more than 120 seconds.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.785348] Not tainted
3.13.0-32-generic #57-Ubuntu
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.801256] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838388] jbd2/sda1-8 D
ffff88103f2d4440 0 322 2 0x00000000
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838396] ffff881022a3bbc8
0000000000000002 ffff88102296dfc0 ffff881022a3bfd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838403] 0000000000014440
0000000000014440 ffff88102296dfc0 ffff88103f2d4cd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838408] ffff88107ffba550
0000000000000002 ffffffff811ee000 ffff881022a3bc40
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838423] Call Trace:
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838438] [<ffffffff811ee000>]
? generic_block_bmap+0x50/0x50
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838447] [<ffffffff817203fd>]
io_schedule+0x9d/0x140
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838468] [<ffffffff811ee00e>]
sleep_on_buffer+0xe/0x20
...
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.838518] INFO: task
qemu-system-x86:26225 blocked for more than 120 seconds.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.884250] Not tainted
3.13.0-32-generic #57-Ubuntu
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.910724] "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968688] qemu-system-x86 D
ffff88103f514440 0 26225 1 0x00000000
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968692] ffff8806707cbd28
0000000000000002 ffff88100e63dfc0 ffff8806707cbfd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968694] 0000000000014440
0000000000014440 ffff88100e63dfc0 ffff88103f514cd8
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968696] ffff88107ffba0e8
0000000000000002 ffffffff8114e190 ffff8806707cbda0
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968699] Call Trace:
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968704] [<ffffffff8114e190>]
? wait_on_page_read+0x60/0x60
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968707] [<ffffffff817203fd>]
io_schedule+0x9d/0x140
Nov 11 20:08:45 l1-jshost-1a-de kernel: [7283447.968715] [<ffffffff8114e19e>]
sleep_on_page+0xe/0x20
"""
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1401181/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
