** Description changed:
- During failover, we observe an oops from device-mapper that renders the
system useless.
+ During failover, we observe an oops from device-mapper that renders the
+ system useless.
+
+
+ Sep 19 15:32:51 ubuntu kernel: [ 4523.872508] phy-0:3: mptsas: ioc0: delete
phy 3, phy-obj (0xffff880166072800)
+ Sep 19 15:32:51 ubuntu kernel: [ 4523.872516] port-0:7: mptsas: ioc0:
delete port 7, sas_addr (0x50015173514ed3ff)
+ Sep 19 15:32:51 ubuntu kernel: [ 4523.874214] mptsas: ioc0: delete expander:
num_phys 25, sas_addr (0x50015173514ed3ff)
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.032281] BUG: unable to handle kernel
NULL pointer dereference at (null)
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.045726] IP: [< (null)>]
(null)
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.054234] PGD 0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.059279] Oops: 0010 [#1] SMP
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.065964] CPU 15
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.068217] Modules linked in: vesafb ghes
lp i7core_edac joydev tpm_tis hed parport ioatdma edac_core dm_round_robin
usbhid hid mptsas mptscsih mptbase igb scsi_transport_sas dca dm_multipath
scsi_dh_alua [last unloaded: scsi_wait_scan]
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083183]
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083186] Pid: 2444, comm: jbd2/dm-5-8
Not tainted 3.0.0-11-generic #18-Ubuntu INTEL MFS5520VI Compute
Module/MFS5520VI Compute Module
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083191] RIP: 0010:[<0000000000000000>]
[< (null)>] (null)
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083195] RSP: 0018:ffff88026321fac8
EFLAGS: 00010082
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083197] RAX: ffff880167006600 RBX:
ffff88016606d7c0 RCX: 0000000000000000
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083199] RDX: 0000000000000002 RSI:
0000000000000001 RDI: ffff88016606d7c0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083201] RBP: ffff88026321fae0 R08:
0000000000000000 R09: 0000000000000001
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083204] R10: 6db6db6db6db6db7 R11:
0000000000000001 R12: ffff88016606d7c0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083206] R13: 0000000000000082 R14:
ffffc90019b7e040 R15: ffff88016606d7c0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083209] FS: 0000000000000000(0000)
GS:ffff88026fce0000(0000) knlGS:0000000000000000
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083212] CS: 0010 DS: 0000 ES: 0000
CR0: 000000008005003b
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083214] CR2: 0000000000000000 CR3:
0000000001a03000 CR4: 00000000000006e0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083217] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083219] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083222] Process jbd2/dm-5-8 (pid:
2444, threadinfo ffff88026321e000, task ffff8802682e4560)
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083224] Stack:
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083226] ffffffff812c93da
ffffc90019b7e040 ffff8802682ccfb8 ffff88026321fb30
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083229] ffffffff812c9533
ffff88026321fb40 000000026606d7c0 ffff88026321fb30
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083233] ffff8802682ccfb8
ffff88016606d7c0 0000000000000082 ffffc90019b7e040
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083236] Call Trace:
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083247] [<ffffffff812c93da>] ?
elv_drain_elevator+0x2a/0x80
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083252] [<ffffffff812c9533>]
__elv_add_request+0x103/0x290
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083258] [<ffffffff812cdaaa>]
blk_insert_cloned_request+0x5a/0x90
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083265] [<ffffffff81499f4e>]
dm_dispatch_request+0x3e/0x70
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083270] [<ffffffff8149b6e1>]
map_request+0xd1/0x140
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083273] [<ffffffff8149b806>]
dm_request_fn+0xb6/0x140
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083277] [<ffffffff812cc6fa>]
queue_unplugged+0x4a/0xe0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083281] [<ffffffff812d0e0d>]
blk_flush_plug_list+0x1ed/0x250
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083284] [<ffffffff812d0e88>]
blk_finish_plug+0x18/0x50
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083292] [<ffffffff81246f97>]
jbd2_journal_commit_transaction+0x307/0x1250
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083299] [<ffffffff8104c82e>] ?
perf_event_task_sched_out+0x2e/0xa0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083303] [<ffffffff8104f232>] ?
update_shares+0xc2/0xf0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083312] [<ffffffff815ea07e>] ?
_raw_spin_lock_irqsave+0x2e/0x40
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083316] [<ffffffff8106f082>] ?
try_to_del_timer_sync+0x92/0x130
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083320] [<ffffffff8124bd3b>]
kjournald2+0xbb/0x220
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083326] [<ffffffff81081c70>] ?
add_wait_queue+0x60/0x60
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083329] [<ffffffff8124bc80>] ?
commit_timeout+0x10/0x10
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083332] [<ffffffff810811cc>]
kthread+0x8c/0xa0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083337] [<ffffffff815f32e4>]
kernel_thread_helper+0x4/0x10
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083341] [<ffffffff81081140>] ?
flush_kthread_worker+0xa0/0xa0
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083344] [<ffffffff815f32e0>] ?
gs_change+0x13/0x13
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083345] Code: Bad RIP value.
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083351] RIP [< (null)>]
(null)
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083354] RSP <ffff88026321fac8>
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.083355] CR2: 0000000000000000
+ Sep 19 15:32:51 ubuntu kernel: [ 4524.098148] ---[ end trace
47a8c4e53f8ecbcd ]---
+ Sep 19 15:32:54 ubuntu kernel: [ 4527.158111] sd 0:0:8:1: alua: port group
00 state A supports toluSnA
+ Sep 19 15:32:55 ubuntu kernel: [ 4528.319360] sd 0:0:8:0: alua: port group
00 state A supports toluSnA
+
+
Easily reproducible:
Using 3 data luns, run iozone stress in a loop with the following options
Share1: iozone -p -K -Z -R -l 2 -u 2 -r 4k -s 100m
Share2: iozone -K -B -D -R -l 2 -u 2 -r 4k -s 50m
Share3: iozone -p -K -B -R -l 2 -u 2 -r 4k -s 50m
Reset SAN storage controller every 4 minutes to force failover, fault
occurs in about 30 mins.
ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image-3.0.0-11-generic 3.0.0-11.18
ProcVersionSignature: Ubuntu 3.0.0-11.18-generic 3.0.4
Uname: Linux 3.0.0-11-generic x86_64
AlsaDevices:
- total 0
- crw-rw---- 1 root audio 116, 1 2011-09-19 16:58 seq
- crw-rw---- 1 root audio 116, 33 2011-09-19 16:58 timer
+ total 0
+ crw-rw---- 1 root audio 116, 1 2011-09-19 16:58 seq
+ crw-rw---- 1 root audio 116, 33 2011-09-19 16:58 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 1.23-0ubuntu1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
Date: Mon Sep 19 17:25:38 2011
HibernationDevice: RESUME=UUID=d12a5c6d-a835-48fd-8f41-3913e9ea07b2
IwConfig: Error: [Errno 2] No such file or directory
MachineType: INTEL MFS5520VI Compute Module
PciMultimedia:
-
+
ProcEnviron:
- LANGUAGE=en_US:
- PATH=(custom, no user)
- LANG=en_US.UTF-8
- SHELL=/bin/bash
+ LANGUAGE=en_US:
+ PATH=(custom, no user)
+ LANG=en_US.UTF-8
+ SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.0.0-11-generic
root=UUID=0ba2d1c6-d5aa-4472-b8c8-300a1d7f914f ro
crashkernel=384M-2G:64M,2G-:128M console=tty0 console=ttyS0,115200n8
RelatedPackageVersions:
- linux-restricted-modules-3.0.0-11-generic N/A
- linux-backports-modules-3.0.0-11-generic N/A
- linux-firmware 1.60
+ linux-restricted-modules-3.0.0-11-generic N/A
+ linux-backports-modules-3.0.0-11-generic N/A
+ linux-firmware 1.60
RfKill: Error: [Errno 2] No such file or directory
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/28/2010
dmi.bios.vendor: Intel Corp.
dmi.bios.version: S5500.86B.01.00.0054.092820101104
dmi.board.name: MFS5520VI Compute Module
dmi.board.vendor: INTEL
dmi.board.version: E55731-402
dmi.chassis.asset.tag: BZVH92400368
dmi.chassis.type: 8
dmi.chassis.vendor: INTEL
dmi.chassis.version: MFS5520VI Compute Module
dmi.modalias:
dmi:bvnIntelCorp.:bvrS5500.86B.01.00.0054.092820101104:bd09/28/2010:svnINTEL:pnMFS5520VIComputeModule:pvr:rvnINTEL:rnMFS5520VIComputeModule:rvrE55731-402:cvnINTEL:ct8:cvrMFS5520VIComputeModule:
dmi.product.name: MFS5520VI Compute Module
dmi.sys.vendor: INTEL
** Attachment added: "serial console logs"
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/854257/+attachment/2428529/+files/screen.c2.b2-console.2.log
** Description changed:
During failover, we observe an oops from device-mapper that renders the
system useless.
-
- Sep 19 15:32:51 ubuntu kernel: [ 4523.872508] phy-0:3: mptsas: ioc0: delete
phy 3, phy-obj (0xffff880166072800)
- Sep 19 15:32:51 ubuntu kernel: [ 4523.872516] port-0:7: mptsas: ioc0:
delete port 7, sas_addr (0x50015173514ed3ff)
- Sep 19 15:32:51 ubuntu kernel: [ 4523.874214] mptsas: ioc0: delete expander:
num_phys 25, sas_addr (0x50015173514ed3ff)
- Sep 19 15:32:51 ubuntu kernel: [ 4524.032281] BUG: unable to handle kernel
NULL pointer dereference at (null)
- Sep 19 15:32:51 ubuntu kernel: [ 4524.045726] IP: [< (null)>]
(null)
- Sep 19 15:32:51 ubuntu kernel: [ 4524.054234] PGD 0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.059279] Oops: 0010 [#1] SMP
- Sep 19 15:32:51 ubuntu kernel: [ 4524.065964] CPU 15
- Sep 19 15:32:51 ubuntu kernel: [ 4524.068217] Modules linked in: vesafb ghes
lp i7core_edac joydev tpm_tis hed parport ioatdma edac_core dm_round_robin
usbhid hid mptsas mptscsih mptbase igb scsi_transport_sas dca dm_multipath
scsi_dh_alua [last unloaded: scsi_wait_scan]
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083183]
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083186] Pid: 2444, comm: jbd2/dm-5-8
Not tainted 3.0.0-11-generic #18-Ubuntu INTEL MFS5520VI Compute
Module/MFS5520VI Compute Module
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083191] RIP: 0010:[<0000000000000000>]
[< (null)>] (null)
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083195] RSP: 0018:ffff88026321fac8
EFLAGS: 00010082
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083197] RAX: ffff880167006600 RBX:
ffff88016606d7c0 RCX: 0000000000000000
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083199] RDX: 0000000000000002 RSI:
0000000000000001 RDI: ffff88016606d7c0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083201] RBP: ffff88026321fae0 R08:
0000000000000000 R09: 0000000000000001
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083204] R10: 6db6db6db6db6db7 R11:
0000000000000001 R12: ffff88016606d7c0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083206] R13: 0000000000000082 R14:
ffffc90019b7e040 R15: ffff88016606d7c0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083209] FS: 0000000000000000(0000)
GS:ffff88026fce0000(0000) knlGS:0000000000000000
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083212] CS: 0010 DS: 0000 ES: 0000
CR0: 000000008005003b
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083214] CR2: 0000000000000000 CR3:
0000000001a03000 CR4: 00000000000006e0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083217] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083219] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083222] Process jbd2/dm-5-8 (pid:
2444, threadinfo ffff88026321e000, task ffff8802682e4560)
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083224] Stack:
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083226] ffffffff812c93da
ffffc90019b7e040 ffff8802682ccfb8 ffff88026321fb30
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083229] ffffffff812c9533
ffff88026321fb40 000000026606d7c0 ffff88026321fb30
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083233] ffff8802682ccfb8
ffff88016606d7c0 0000000000000082 ffffc90019b7e040
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083236] Call Trace:
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083247] [<ffffffff812c93da>] ?
elv_drain_elevator+0x2a/0x80
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083252] [<ffffffff812c9533>]
__elv_add_request+0x103/0x290
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083258] [<ffffffff812cdaaa>]
blk_insert_cloned_request+0x5a/0x90
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083265] [<ffffffff81499f4e>]
dm_dispatch_request+0x3e/0x70
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083270] [<ffffffff8149b6e1>]
map_request+0xd1/0x140
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083273] [<ffffffff8149b806>]
dm_request_fn+0xb6/0x140
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083277] [<ffffffff812cc6fa>]
queue_unplugged+0x4a/0xe0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083281] [<ffffffff812d0e0d>]
blk_flush_plug_list+0x1ed/0x250
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083284] [<ffffffff812d0e88>]
blk_finish_plug+0x18/0x50
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083292] [<ffffffff81246f97>]
jbd2_journal_commit_transaction+0x307/0x1250
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083299] [<ffffffff8104c82e>] ?
perf_event_task_sched_out+0x2e/0xa0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083303] [<ffffffff8104f232>] ?
update_shares+0xc2/0xf0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083312] [<ffffffff815ea07e>] ?
_raw_spin_lock_irqsave+0x2e/0x40
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083316] [<ffffffff8106f082>] ?
try_to_del_timer_sync+0x92/0x130
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083320] [<ffffffff8124bd3b>]
kjournald2+0xbb/0x220
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083326] [<ffffffff81081c70>] ?
add_wait_queue+0x60/0x60
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083329] [<ffffffff8124bc80>] ?
commit_timeout+0x10/0x10
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083332] [<ffffffff810811cc>]
kthread+0x8c/0xa0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083337] [<ffffffff815f32e4>]
kernel_thread_helper+0x4/0x10
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083341] [<ffffffff81081140>] ?
flush_kthread_worker+0xa0/0xa0
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083344] [<ffffffff815f32e0>] ?
gs_change+0x13/0x13
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083345] Code: Bad RIP value.
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083351] RIP [< (null)>]
(null)
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083354] RSP <ffff88026321fac8>
- Sep 19 15:32:51 ubuntu kernel: [ 4524.083355] CR2: 0000000000000000
- Sep 19 15:32:51 ubuntu kernel: [ 4524.098148] ---[ end trace
47a8c4e53f8ecbcd ]---
- Sep 19 15:32:54 ubuntu kernel: [ 4527.158111] sd 0:0:8:1: alua: port group
00 state A supports toluSnA
- Sep 19 15:32:55 ubuntu kernel: [ 4528.319360] sd 0:0:8:0: alua: port group
00 state A supports toluSnA
+ [ 4883.223130] phy-0:2: mptsas: ioc0: delete phy 2, phy-obj
(0xffff880268fe2400)
+ [ 4883.251641] port-0:8: mptsas: ioc0: delete port 8, sas_addr
(0x50015173514ed2ff)
+ [ 4883.283772] mptsas: ioc0: delete expander: num_phys 25, sas_addr
(0x50015173514ed2ff)
+ [ 4885.089886] BUG: unable to handle kernel NULL pointer dereference at
(null)
+ [ 4885.122648] IP: [< (null)>] (null)
+ [ 4885.140264] PGD 0
+ [ 4885.154573] Oops: 0010 [#2] SMP
+ [ 4885.169357] CPU 0
+ [ 4885.171522] Modules linked in: vesafb ghes lp i7core_edac joydev tpm_tis
hed parport ioatdma edac_core dm_round_robin usbhid hid mptsas mptscsih mptbase
igb scsi_transport_sas dca dm_multipath scsi_dh_alua [last unloaded:
scsi_wait_scan]
+ [ 4885.239936]
+ [ 4885.252092] Pid: 1634, comm: flush-253:7 Tainted: G D
3.0.0-11-generic #18-Ubuntu INTEL MFS5520VI Compute Module/MFS5520VI Compute
Module
+ [ 4885.289901] RIP: 0010:[<0000000000000000>] [< (null)>]
(null)
+ [ 4885.321165] RSP: 0018:ffff880265ca5598 EFLAGS: 00010086
+ [ 4885.338648] RAX: ffff880165c9ab40 RBX: ffff88016606e540 RCX:
0000000000000000
+ [ 4885.358274] RDX: 0000000000000002 RSI: 0000000000000001 RDI:
ffff88016606e540
+ [ 4885.377471] RBP: ffff880265ca55b0 R08: 0000000000000000 R09:
0000000000000001
+ [ 4885.396685] R10: 6db6db6db6db6db7 R11: 0000000000000001 R12:
ffff88016606e540
+ [ 4885.415695] R13: 0000000000000086 R14: ffffc9001a5d7040 R15:
ffff88016606e540
+ [ 4885.434501] FS: 0000000000000000(0000) GS:ffff88016fc00000(0000)
knlGS:0000000000000000
+ [ 4885.465293] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
+ [ 4885.483058] CR2: 0000000000000000 CR3: 0000000001a03000 CR4:
00000000000006f0
+ [ 4885.502813] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
+ [ 4885.521729] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
+ [ 4885.540344] Process flush-253:7 (pid: 1634, threadinfo ffff880265ca4000,
task ffff8802630a2e40)
+ [ 4885.570886] Stack:
+ [ 4885.583169] ffffffff812c93da ffffc9001a5d7040 ffff8801674c04c8
ffff880265ca5600
+ [ 4885.611719] ffffffff812c9533 ffff880265ca5610 000000026606e540
ffff880265ca5600
+ [ 4885.641208] ffff8801674c04c8 ffff88016606e540 0000000000000086
ffffc9001a5d7040
+ [ 4885.671757] Call Trace:
+ [ 4885.685577] [<ffffffff812c93da>] ? elv_drain_elevator+0x2a/0x80
+ [ 4885.703001] [<ffffffff812c9533>] __elv_add_request+0x103/0x290
+ [ 4885.719536] [<ffffffff812cdaaa>] blk_insert_cloned_request+0x5a/0x90
+ [ 4885.737058] [<ffffffff81499f4e>] dm_dispatch_request+0x3e/0x70
+ [ 4885.754074] [<ffffffff8149b6e1>] map_request+0xd1/0x140
+ [ 4885.769622] [<ffffffff8149b806>] dm_request_fn+0xb6/0x140
+ [ 4885.785586] [<ffffffff812d0bfd>] __make_request+0x32d/0x350
+ [ 4885.801901] [<ffffffff8149af96>] dm_request+0x36/0x40
+ [ 4885.817725] [<ffffffff812cdd2f>] generic_make_request.part.51+0x24f/0x510
+ [ 4885.835224] [<ffffffff812ce035>] generic_make_request+0x45/0x60
+ [ 4885.851707] [<ffffffff812ce0d7>] submit_bio+0x87/0x110
+ [ 4885.866729] [<ffffffff8120c974>] ? ext4_bio_write_page+0x134/0x180
+ [ 4885.883141] [<ffffffff8120c639>] ext4_io_submit+0x29/0x60
+ [ 4885.898853] [<ffffffff81206821>] mpage_da_submit_io+0x311/0x580
+ [ 4885.914641] [<ffffffff81115575>] ? pagevec_lookup_tag+0x25/0x40
+ [ 4885.930815] [<ffffffff8120ae94>] ? write_cache_pages_da+0x144/0x440
+ [ 4885.947078] [<ffffffff8120aad6>] mpage_da_map_and_submit+0x1d6/0x360
+ [ 4885.963793] [<ffffffff81244903>] ? jbd2_journal_start+0x13/0x20
+ [ 4885.979979] [<ffffffff812214e9>] ? ext4_journal_start_sb+0x69/0x170
+ [ 4885.996763] [<ffffffff812cc706>] ? queue_unplugged+0x56/0xe0
+ [ 4886.012142] [<ffffffff8120b4bc>] ext4_da_writepages+0x32c/0x5e0
+ [ 4886.027705] [<ffffffff812e7d16>] ? cpumask_next_and+0x36/0x50
+ [ 4886.042877] [<ffffffff81114a81>] do_writepages+0x21/0x40
+ [ 4886.057819] [<ffffffff8118edf3>] writeback_single_inode+0x103/0x270
+ [ 4886.073612] [<ffffffff8118f201>] writeback_sb_inodes+0xe1/0x1b0
+ [ 4886.088698] [<ffffffff8118f566>] writeback_inodes_wb+0xa6/0xf0
+ [ 4886.103542] [<ffffffff8118f8df>] wb_writeback+0x32f/0x430
+ [ 4886.117531] [<ffffffff8118fa78>] wb_check_old_data_flush+0x98/0xa0
+ [ 4886.132529] [<ffffffff8118fbd1>] wb_do_writeback+0x151/0x1f0
+ [ 4886.146830] [<ffffffff8106dfc0>] ? usleep_range+0x50/0x50
+ [ 4886.160352] [<ffffffff8118fcf3>] bdi_writeback_thread+0x83/0x2a0
+ [ 4886.175146] [<ffffffff8118fc70>] ? wb_do_writeback+0x1f0/0x1f0
+ [ 4886.188875] [<ffffffff810811cc>] kthread+0x8c/0xa0
+ [ 4886.202035] [<ffffffff815f32e4>] kernel_thread_helper+0x4/0x10
+ [ 4886.216271] [<ffffffff81081140>] ? flush_kthread_worker+0xa0/0xa0
+ [ 4886.230738] [<ffffffff815f32e0>] ? gs_change+0x13/0x13
+ [ 4886.244335] Code: Bad RIP value.
+ [ 4886.255626] RIP [< (null)>] (null)
+ [ 4886.268564] RSP <ffff880265ca5598>
+ [ 4886.279681] CR2: 0000000000000000
+ [ 4886.291223] ---[ end trace 47a8c4e53f8ecbce ]---
Easily reproducible:
Using 3 data luns, run iozone stress in a loop with the following options
Share1: iozone -p -K -Z -R -l 2 -u 2 -r 4k -s 100m
Share2: iozone -K -B -D -R -l 2 -u 2 -r 4k -s 50m
Share3: iozone -p -K -B -R -l 2 -u 2 -r 4k -s 50m
Reset SAN storage controller every 4 minutes to force failover, fault
occurs in about 30 mins.
ProblemType: Bug
DistroRelease: Ubuntu 11.10
Package: linux-image-3.0.0-11-generic 3.0.0-11.18
ProcVersionSignature: Ubuntu 3.0.0-11.18-generic 3.0.4
Uname: Linux 3.0.0-11-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 2011-09-19 16:58 seq
crw-rw---- 1 root audio 116, 33 2011-09-19 16:58 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 1.23-0ubuntu1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
CRDA: Error: [Errno 2] No such file or directory
Date: Mon Sep 19 17:25:38 2011
HibernationDevice: RESUME=UUID=d12a5c6d-a835-48fd-8f41-3913e9ea07b2
IwConfig: Error: [Errno 2] No such file or directory
MachineType: INTEL MFS5520VI Compute Module
PciMultimedia:
ProcEnviron:
LANGUAGE=en_US:
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-3.0.0-11-generic
root=UUID=0ba2d1c6-d5aa-4472-b8c8-300a1d7f914f ro
crashkernel=384M-2G:64M,2G-:128M console=tty0 console=ttyS0,115200n8
RelatedPackageVersions:
linux-restricted-modules-3.0.0-11-generic N/A
linux-backports-modules-3.0.0-11-generic N/A
linux-firmware 1.60
RfKill: Error: [Errno 2] No such file or directory
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 09/28/2010
dmi.bios.vendor: Intel Corp.
dmi.bios.version: S5500.86B.01.00.0054.092820101104
dmi.board.name: MFS5520VI Compute Module
dmi.board.vendor: INTEL
dmi.board.version: E55731-402
dmi.chassis.asset.tag: BZVH92400368
dmi.chassis.type: 8
dmi.chassis.vendor: INTEL
dmi.chassis.version: MFS5520VI Compute Module
dmi.modalias:
dmi:bvnIntelCorp.:bvrS5500.86B.01.00.0054.092820101104:bd09/28/2010:svnINTEL:pnMFS5520VIComputeModule:pvr:rvnINTEL:rnMFS5520VIComputeModule:rvrE55731-402:cvnINTEL:ct8:cvrMFS5520VIComputeModule:
dmi.product.name: MFS5520VI Compute Module
dmi.sys.vendor: INTEL
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/854257
Title:
device-mapper oops during failover, all I/O hangs
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/854257/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
