Upgraded to 3.6 today on my dev box and after seeing an FC attached SAN
go down and come back up (due to an expected reboot) I started getting
the following in my logs. It continues even after the array is back and
functioning - I'm seeing:
kernel:[109104.348034] BUG: soft lockup - CPU#6 stuck for 23s!
[kworker/6:0:30692]
repeated on logged in sessions and backtraces like the following (this
is the first). I don't see the same problem under 3.5.
[10819.389706] device-mapper: multipath: Failing path 8:240.
[11233.683936] device-mapper: multipath: Failing path 8:240.
[108394.592042] rport-10:0-4: blocked FC remote port time out: removing target
and saving binding
[108394.609594] sd 10:0:1:0: rejecting I/O to offline device
[108394.620457] lpfc 0000:0c:00.0: 2:(0):0203 Devloss timeout on WWPN
21:11:00:02:ac:01:86:06 NPort x030500 Data: x0 x7 x0
[108394.620591] sd 10:0:1:0: alua: Detached
[108394.650159] sd 10:0:1:0: [sdbc] Synchronizing SCSI cache
[108394.661071] sd 10:0:1:0: [sdbc]
[108394.667877] Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
[108394.680154] ses 10:0:1:254: alua: Detached
[108420.348032] BUG: soft lockup - CPU#6 stuck for 23s! [kworker/6:0:30692]
[108420.352003] Modules linked in: nfsv4 autofs4 ip6table_filter ip6_tables
iptable_filter ip_tables ebtable_nat ebtables x_tables rpcsec_gss_krb5 ipv6
nfsd nfs_acl auth_rpcgss nfs lockd sunrpc dm_round_robin dm_multipath
ipmi_devintf ipmi_si ipmi_msghandler sg evdev acpi_cpufreq freq_table serio_raw
mperf processor button thermal_sys coretemp kvm_intel kvm lpc_ich ioatdma
mfd_core tpm_tis i2c_i801 tpm microcode tpm_bios rng_core i2c_core i5k_amb dca
ses enclosure ata_generic lpfc ata_piix scsi_transport_fc scsi_tgt
[108420.352003] CPU 6
[108420.352003] Pid: 30692, comm: kworker/6:0 Not tainted 3.6.0 #5 Intel
S5000PAL./S5000PAL0
[108420.352003] RIP: 0010:[<ffffffff8134e744>] [<ffffffff8134e744>]
_raw_spin_unlock_irqrestore+0x5/0x6
[108420.352003] RSP: 0018:ffff8802563a7d98 EFLAGS: 00000286
[108420.352003] RAX: ffff88024e975000 RBX: 00000000000000bb RCX:
0000000000000000
[108420.352003] RDX: 0000000000000000 RSI: 0000000000000286 RDI:
ffff88024e975050
[108420.352003] RBP: ffff88024e975000 R08: 0000000000000000 R09:
ffffffff8166f890
[108420.352003] R10: ffff88024e975000 R11: ffffffffa00d6bf0 R12:
0000000000000000
[108420.352003] R13: ffffffff8166f890 R14: ffff88024e975000 R15:
ffffffffa00d6bf0
[108420.352003] FS: 0000000000000000(0000) GS:ffff88025fd80000(0000)
knlGS:0000000000000000
[108420.352003] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[108420.352003] CR2: 00007f5b5dec6070 CR3: 000000024f0eb000 CR4:
00000000000007e0
[108420.352003] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[108420.352003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[108420.352003] Process kworker/6:0 (pid: 30692, threadinfo ffff8802563a6000,
task ffff880252f32a10)
[108420.352003] Stack:
[108420.352003] ffffffff8125a498 ffff88025fd8d080 0000000000000286
ffffffffa0015c28
[108420.352003] ffff88024d1207c0 ffff88025fd8d080 ffff88025fd98100
ffffffffa0015c28
[108420.352003] 0000000000000000 ffff88024f22abd8 ffffffff81045d07
0000000000012240
[108420.352003] Call Trace:
[108420.352003] [<ffffffff8125a498>] ? scsi_remove_target+0x138/0x154
[108420.352003] [<ffffffffa0015c28>] ? store_fc_host_system_hostname+0x66/0x66
[scsi_transport_fc]
[108420.352003] [<ffffffffa0015c28>] ? store_fc_host_system_hostname+0x66/0x66
[scsi_transport_fc]
[108420.352003] [<ffffffff81045d07>] ? process_one_work+0x1f8/0x30a
[108420.352003] [<ffffffff81046034>] ? worker_thread+0x21b/0x314
[108420.352003] [<ffffffff81045e19>] ? process_one_work+0x30a/0x30a
[108420.352003] [<ffffffff81045e19>] ? process_one_work+0x30a/0x30a
[108420.352003] [<ffffffff810496cf>] ? kthread+0x81/0x89
[108420.352003] [<ffffffff81350174>] ? kernel_thread_helper+0x4/0x10
[108420.352003] [<ffffffff8104964e>] ? kthread_freezable_should_stop+0x4e/0x4e
[108420.352003] [<ffffffff81350170>] ? gs_change+0xb/0xb
[108420.352003] Code: 66 39 d0 0f 94 c0 0f b6 c0 c3 fa b8 00 01 00 00 f0 66 0f
c1 07 88 c2 66 c1 e8 08 38 c2 74 06 f3 90 8a 17 eb f6 c3 80 07 01 56 9d <c3> 83
ca ff f0 0f c1 17 b8 01 00 00 00 ff ca 79 05 f0 ff 07 30
[108448.348033] BUG: soft lockup - CPU#6 stuck for 22s! [kworker/6:0:30692]
[108448.352003] Modules linked in: nfsv4 autofs4 ip6table_filter ip6_tables
iptable_filter ip_tables ebtable_nat ebtables x_tables rpcsec_gss_krb5 ipv6
nfsd nfs_acl auth_rpcgss nfs lockd sunrpc dm_round_robin dm_multipath
ipmi_devintf ipmi_si ipmi_msghandler sg evdev acpi_cpufreq freq_table serio_raw
mperf processor button thermal_sys coretemp kvm_intel kvm lpc_ich ioatdma
mfd_core tpm_tis i2c_i801 tpm microcode tpm_bios rng_core i2c_core i5k_amb dca
ses enclosure ata_generic lpfc ata_piix scsi_transport_fc scsi_tgt
[108448.352003] CPU 6
[108448.352003] Pid: 30692, comm: kworker/6:0 Not tainted 3.6.0 #5 Intel
S5000PAL./S5000PAL0
[108448.352003] RIP: 0010:[<ffffffff8134e744>] [<ffffffff8134e744>]
_raw_spin_unlock_irqrestore+0x5/0x6
[108448.352003] RSP: 0018:ffff8802563a7d98 EFLAGS: 00000286
[108448.352003] RAX: ffff88024e975000 RBX: 00000000000000a2 RCX:
0000000000000087
[108448.352003] RDX: 0000000000000087 RSI: 0000000000000286 RDI:
ffff88024e975050
[108448.352003] RBP: ffff88024e975000 R08: 0000000000000000 R09:
ffffffff8166f890
[108448.352003] R10: ffff88024e975000 R11: ffffffffa00d6bf0 R12:
0000000000000000
[108448.352003] R13: ffffffff8166f890 R14: ffff88024e975000 R15:
ffffffffa00d6bf0
[108448.352003] FS: 0000000000000000(0000) GS:ffff88025fd80000(0000)
knlGS:0000000000000000
[108448.352003] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[108448.352003] CR2: 00007f5b5dec6070 CR3: 000000024f0eb000 CR4:
00000000000007e0
[108448.352003] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[108448.352003] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[108448.352003] Process kworker/6:0 (pid: 30692, threadinfo ffff8802563a6000,
task ffff880252f32a10)
[108448.352003] Stack:
[108448.352003] ffffffff8125a498 ffff88025fd8d080 0000000000000286
ffffffffa0015c28
[108448.352003] ffff88024d1207c0 ffff88025fd8d080 ffff88025fd98100
ffffffffa0015c28
[108448.352003] 0000000000000000 ffff88024f22abd8 ffffffff81045d07
0000000000012240
[108448.352003] Call Trace:
[108448.352003] [<ffffffff8125a498>] ? scsi_remove_target+0x138/0x154
[108448.352003] [<ffffffffa0015c28>] ? store_fc_host_system_hostname+0x66/0x66
[scsi_transport_fc]
[108448.352003] [<ffffffffa0015c28>] ? store_fc_host_system_hostname+0x66/0x66
[scsi_transport_fc]
[108448.352003] [<ffffffff81045d07>] ? process_one_work+0x1f8/0x30a
[108448.352003] [<ffffffff81046034>] ? worker_thread+0x21b/0x314
[108448.352003] [<ffffffff81045e19>] ? process_one_work+0x30a/0x30a
[108448.352003] [<ffffffff81045e19>] ? process_one_work+0x30a/0x30a
[108448.352003] [<ffffffff810496cf>] ? kthread+0x81/0x89
[108448.352003] [<ffffffff81350174>] ? kernel_thread_helper+0x4/0x10
[108448.352003] [<ffffffff8104964e>] ? kthread_freezable_should_stop+0x4e/0x4e
[108448.352003] [<ffffffff81350170>] ? gs_change+0xb/0xb
[108448.352003] Code: 66 39 d0 0f 94 c0 0f b6 c0 c3 fa b8 00 01 00 00 f0 66 0f
c1 07 88 c2 66 c1 e8 08 38 c2 74 06 f3 90 8a 17 eb f6 c3 80 07 01 56 9d <c3> 83
ca ff f0 0f c1 17 b8 01 00 00 00 ff ca 79 05 f0 ff 07 30
J.
--
101 things you can't have too | .''`. Debian GNU/Linux Developer
much of : 28 - Sunshine. | : :' : Happy to accept PGP signed
| `. `' or encrypted mail - RSA
| `- key on the keyservers.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
