Pretty fed up with people saying this could be so many different issues.
So much so that i finally decided to risk my data to prove it.... read
the following.
***___This has got to be the card / chipset / sata_mv driver._____***
Short and simple version of my issues:
- This does not depend on drive types
- Appears to be caused by MV88SX6081 chipset
- Could be a problem in SATA_MV driver
- I need replacement controller suggestions
Details to all non believers (it’s not a power / hardware issue):
I moved 5 of the 7 drives to my onboard controller (have 6 sata ports on the
mobo, last was used by the system drive).
Left 2 of the western digital drives on the MV88SX6081 8-port SATA II:
- sdg
- sdh
After the advice of some through email, I unplugged everything that wasn't
needed. They assumed that it could have been power giving the number of drives
I had in the machine. What was left on a tx750w corsair power supply:
- mobo (c2d, 4gb ram)
- 7 sata raid drives - spread across multiple power supply rails
- 1 sata system drive
- Super Micro SAT2-MV8 (MV88SX6081 8-port SATA II)
- intel pcie 10/100/1000 network card
Then I replaced the sate cables 1 more time with old cables I knew worked. I
also threw in the brand new controller card as well (have a few spares lying
around).
I brought everything up and upgraded to:
Then I started to rebuild the raid. Everything went fine, no freezes.
**This was the first indication that this only happens under heavy load on
multiple ports as has been brought up before.
So then I started copying data over. About 180GB's the card hard reset both of
the drives attached to it and knocked them both out of the raid.
**This was also significantly different from before when I was utilizing all
the ports as it seemed to work great for quite some time, it wasn't until I was
well into the process that the card finally gave up.
See the attached dmesg and /var/log/messages. This is the 2nd time I’ve had
this card degrade my raid and almost give me a heart attack.
The cards are going in the trash at this point. I'm open to suggestions as to
possibly replacement. I don’t need a hardware raid card, just a decent
controller with great *nix support and lots of ports.
::sigh:: I don’t know who to contact but this is the end of the line for me
with this controller and hopefully my issues.
Attempting to get my data back as we speak with 2 failed drives in a
raid 5... wonderful times.
dmsg of the event:
[ 1061.040118] md: recovery of RAID array md1
[ 1061.040120] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 1061.040122] md: using maximum available idle IO bandwidth (but not more than
200000 KB/sec) for recovery.
[ 1061.040126] md: using 128k window, over a total of 488383744 blocks.
[11208.852220] md: md1: recovery done.
[11209.020072] RAID5 conf printout:
[11209.020076] --- rd:7 wd:7
[11209.020079] disk 0, o:1, dev:sdd1
[11209.020080] disk 1, o:1, dev:sdb1
[11209.020081] disk 2, o:1, dev:sdh1
[11209.020082] disk 3, o:1, dev:sdc1
[11209.020083] disk 4, o:1, dev:sdf1
[11209.020084] disk 5, o:1, dev:sde1
[11209.020085] disk 6, o:1, dev:sdg1
[19844.431690] SGI XFS with ACLs, security attributes, realtime, large
block/inode numbers, no debug enabled
[19844.433148] SGI XFS Quota Management subsystem
[19844.442507] Filesystem "md1": Disabling barriers, trial barrier write failed
[19844.442658] XFS mounting filesystem md1
[19844.893398] Ending clean XFS mount for filesystem: md1
[27027.170016] ata5.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[27027.170041] ata5.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
[27027.170041] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask 0x4
(timeout)
[27027.170083] ata5.00: status: { DRDY }
[27027.170099] ata5: hard resetting link
[27027.680034] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[27027.720050] ata5.00: max_sectors limited to 256 for NCQ
[27027.780047] ata5.00: max_sectors limited to 256 for NCQ
[27027.780050] ata5.00: configured for UDMA/133
[27027.780055] end_request: I/O error, dev sdg, sector 73
[27027.780073] md: super_written gets error=-5, uptodate=0
[27027.780076] raid5: Disk failure on sdg1, disabling device.
[27027.780077] raid5: Operation continuing on 6 devices.
[27027.780117] ata5: EH complete
[27027.780674] sd 4:0:0:0: [sdg] 976773168 512-byte hardware sectors (500108 MB)
[27027.780800] sd 4:0:0:0: [sdg] Write Protect is off
[27027.780803] sd 4:0:0:0: [sdg] Mode Sense: 00 3a 00 00
[27027.781038] sd 4:0:0:0: [sdg] Write cache: enabled, read cache: enabled,
doesn't support DPO or FUA
[27057.930015] ata12.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
[27057.930039] ata12.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
[27057.930040] res 40/00:00:01:4f:c2/00:00:00:00:00/00 Emask 0x4
(timeout)
[27057.930081] ata12.00: status: { DRDY }
[27057.930098] ata12: hard resetting link
[27058.440033] ata12: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[27058.480049] ata12.00: max_sectors limited to 256 for NCQ
[27058.540047] ata12.00: max_sectors limited to 256 for NCQ
[27058.540050] ata12.00: configured for UDMA/133
[27058.540055] end_request: I/O error, dev sdh, sector 71
[27058.540072] md: super_written gets error=-5, uptodate=0
[27058.540075] raid5: Disk failure on sdh1, disabling device.
[27058.540076] raid5: Operation continuing on 5 devices.
[27058.540113] ata12: EH complete
[27058.540754] sd 11:0:0:0: [sdh] 976773168 512-byte hardware sectors (500108
MB)
[27058.540879] sd 11:0:0:0: [sdh] Write Protect is off
[27058.540882] sd 11:0:0:0: [sdh] Mode Sense: 00 3a 00 00
[27058.541070] sd 11:0:0:0: [sdh] Write cache: enabled, read cache: enabled,
doesn't support DPO or FUA
[27058.584017] RAID5 conf printout:
[27058.584020] --- rd:7 wd:5
[27058.584022] disk 0, o:1, dev:sdd1
[27058.584023] disk 1, o:1, dev:sdb1
[27058.584024] disk 2, o:0, dev:sdh1
[27058.584025] disk 3, o:1, dev:sdc1
[27058.584027] disk 4, o:1, dev:sdf1
[27058.584028] disk 5, o:1, dev:sde1
[27058.584029] disk 6, o:0, dev:sdg1
[27061.521245] BUG: soft lockup - CPU#1 stuck for 61s! [smbd:28171]
[27061.521251] Modules linked in: xfs aes_x86_64 aes_generic ecb
crypto_blkcipher ecryptfs ipv6 af_packet iptable_filter ip_tables x_tables ac
sbp2 parport_pc lp parport loop psmouse pcspkr serio_raw iTCO_wdt
iTCO_vendor_support evdev button intel_agp snd_hda_intel snd_pcm shpchp
snd_timer pci_hotplug snd soundcore snd_page_alloc ext3 jbd mbcache sd_mod
crc_t10dif sg pata_acpi pata_marvell usbhid hid ohci1394 ieee1394 sata_mv
ata_generic ata_piix libata scsi_mod dock sky2 e1000e ehci_hcd uhci_hcd usbcore
raid10 raid456 async_xor async_memcpy async_tx xor raid1 raid0 multipath linear
md_mod dm_mirror dm_log dm_snapshot dm_mod thermal processor fan fbcon tileblit
font bitblit softcursor fuse
[27061.521251] CPU 1:
[27061.521251] Modules linked in: xfs aes_x86_64 aes_generic ecb
crypto_blkcipher ecryptfs ipv6 af_packet iptable_filter ip_tables x_tables ac
sbp2 parport_pc lp parport loop psmouse pcspkr serio_raw iTCO_wdt
iTCO_vendor_support evdev button intel_agp snd_hda_intel snd_pcm shpchp
snd_timer pci_hotplug snd soundcore snd_page_alloc ext3 jbd mbcache sd_mod
crc_t10dif sg pata_acpi pata_marvell usbhid hid ohci1394 ieee1394 sata_mv
ata_generic ata_piix libata scsi_mod dock sky2 e1000e ehci_hcd uhci_hcd usbcore
raid10 raid456 async_xor async_memcpy async_tx xor raid1 raid0 multipath linear
md_mod dm_mirror dm_log dm_snapshot dm_mod thermal processor fan fbcon tileblit
font bitblit softcursor fuse
[27061.521251] Pid: 28171, comm: smbd Not tainted 2.6.27-9-server #1
[27061.521251] RIP: 0010:[<ffffffff802abf0c>] [<ffffffff802abf0c>]
find_get_pages+0x6c/0x110
[27061.521251] RSP: 0018:ffff880129453358 EFLAGS: 00000246
[27061.521251] RAX: ffff880128d89330 RBX: ffff880129453398 RCX: 0000000000000002
[27061.521251] RDX: 0000000000000003 RSI: 0000000000000000 RDI: ffffe200022e9e80
[27061.521251] RBP: ffff880129453308 R08: ffffe200009df6c8 R09: 0000000000000005
[27061.521251] R10: 0000000000000037 R11: 00000000001c5778 R12: ffffffff802b6b29
[27061.521251] R13: ffff880123a107d0 R14: ffffe20001c6f6c0 R15: 0000000000000286
[27061.521251] FS: 00007fb72cdf6700(0000) GS:ffff88012fc02980(0000)
knlGS:0000000000000000
[27061.521251] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[27061.521251] CR2: 00007f1648629000 CR3: 000000012956d000 CR4: 00000000000006e0
[27061.521251] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[27061.521251] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[27061.521251]
[27061.521251] Call Trace:
[27061.521251] [<ffffffff802abee3>] ? find_get_pages+0x43/0x110
[27061.521251] [<ffffffff802b6984>] ? pagevec_lookup+0x24/0x30
[27061.521251] [<ffffffffa04e100d>] ? xfs_cluster_write+0xad/0x180 [xfs]
[27061.521251] [<ffffffffa04e1578>] ? xfs_page_state_convert+0x498/0x760 [xfs]
[27061.521251] [<ffffffffa04e19a1>] ? xfs_vm_writepage+0x71/0x120 [xfs]
[27061.521251] [<ffffffff802b9274>] ? pageout+0x124/0x270
[27061.521251] [<ffffffff802ab06a>] ? page_waitqueue+0xa/0x90
[27061.521251] [<ffffffff802b986d>] ? shrink_page_list+0x34d/0x530
[27061.521251] [<ffffffff802b8e49>] ? __isolate_lru_page+0x79/0xb0
[27061.521251] [<ffffffff802b8f0a>] ? isolate_lru_pages+0x8a/0x220
[27061.521251] [<ffffffff802b9bf2>] ? shrink_inactive_list+0x1a2/0x4b0
[27061.521251] [<ffffffff802b9f7b>] ? shrink_zone+0x7b/0x160
[27061.521251] [<ffffffff802ba0ed>] ? shrink_zones+0x8d/0x150
[27061.521251] [<ffffffff802ba236>] ? do_try_to_free_pages+0x86/0x2e0
[27061.521251] [<ffffffff802ba587>] ? try_to_free_pages+0x67/0x70
[27061.521251] [<ffffffff802b90a0>] ? isolate_pages_global+0x0/0x50
[27061.521251] [<ffffffff802b28b1>] ? __alloc_pages_internal+0x241/0x510
[27061.521251] [<ffffffff802d565d>] ? alloc_pages_current+0xad/0x110
[27061.521251] [<ffffffff802ac477>] ? __page_cache_alloc+0x67/0x80
[27061.521251] [<ffffffff802ad0b3>] ? __grab_cache_page+0x63/0xb0
[27061.521251] [<ffffffff80316a59>] ? block_write_begin+0x89/0xf0
[27061.521251] [<ffffffffa04e04ca>] ? xfs_vm_write_begin+0x2a/0x30 [xfs]
[27061.521251] [<ffffffffa04e0040>] ? xfs_get_blocks+0x0/0x20 [xfs]
[27061.521251] [<ffffffff802ab7ac>] ? generic_perform_write+0xbc/0x1c0
[27061.521251] [<ffffffff802ad512>] ? generic_file_buffered_write+0x92/0x170
[27061.521251] [<ffffffffa04e92d3>] ? xfs_write+0x6b3/0x9b0 [xfs]
[27061.521251] [<ffffffff80385a69>] ? apparmor_socket_recvmsg+0x19/0x20
[27061.521251] [<ffffffff803aaf70>] ? memset_c+0x20/0x30
[27061.521251] [<ffffffffa04e4c88>] ? xfs_file_aio_write+0x58/0x60 [xfs]
[27061.521251] [<ffffffff802e9559>] ? do_sync_write+0xf9/0x140
[27061.521251] [<ffffffff802e9699>] ? do_sync_read+0xf9/0x140
[27061.521251] [<ffffffff80266fb0>] ? autoremove_wake_function+0x0/0x40
[27061.521251] [<ffffffff80386821>] ? aa_file_permission+0x21/0xf0
[27061.521251] [<ffffffff80386948>] ? apparmor_file_permission+0x28/0x30
[27061.521251] [<ffffffff803613e6>] ? security_file_permission+0x16/0x20
[27061.521251] [<ffffffff802e9c1b>] ? vfs_write+0xcb/0x130
[27061.521251] [<ffffffff802e9d1a>] ? sys_pwrite64+0x9a/0xa0
[27061.521251] [<ffffffff8021285a>] ? system_call_fastpath+0x16/0x1b
[27061.521251]
[27095.080066] RAID5 conf printout:
[27095.080071] --- rd:7 wd:5
[27095.080074] disk 0, o:1, dev:sdd1
[27095.080076] disk 1, o:1, dev:sdb1
[27095.080077] disk 2, o:0, dev:sdh1
[27095.080079] disk 3, o:1, dev:sdc1
[27095.080080] disk 4, o:1, dev:sdf1
[27095.080082] disk 5, o:1, dev:sde1
[27095.080090] RAID5 conf printout:
[27095.080091] --- rd:7 wd:5
[27095.080092] disk 0, o:1, dev:sdd1
[27095.080093] disk 1, o:1, dev:sdb1
[27095.080094] disk 2, o:0, dev:sdh1
[27095.080095] disk 3, o:1, dev:sdc1
[27095.080097] disk 4, o:1, dev:sdf1
[27095.080098] disk 5, o:1, dev:sde1
[27095.140011] RAID5 conf printout:
[27095.140017] --- rd:7 wd:5
[27095.140019] disk 0, o:1, dev:sdd1
[27095.140022] disk 1, o:1, dev:sdb1
[27095.140024] disk 3, o:1, dev:sdc1
[27095.140026] disk 4, o:1, dev:sdf1
[27095.140027] disk 5, o:1, dev:sde1
[27095.140511] Buffer I/O error on device md1, logical block 455870845
[27095.140545] lost page write due to I/O error on md1
[27095.140550] Buffer I/O error on device md1, logical block 455870846
[27095.140567] lost page write due to I/O error on md1
[27095.140569] Buffer I/O error on device md1, logical block 455870847
[27095.140585] lost page write due to I/O error on md1
[27095.140587] Buffer I/O error on device md1, logical block 455870848
[27095.140604] lost page write due to I/O error on md1
[27095.140606] Buffer I/O error on device md1, logical block 455870849
[27095.140622] lost page write due to I/O error on md1
[27095.140624] Buffer I/O error on device md1, logical block 455870850
[27095.140641] lost page write due to I/O error on md1
[27095.140642] Buffer I/O error on device md1, logical block 455870851
[27095.140659] lost page write due to I/O error on md1
[27095.140661] Buffer I/O error on device md1, logical block 455870852
[27095.140677] lost page write due to I/O error on md1
[27095.140679] Buffer I/O error on device md1, logical block 455870853
[27095.140696] lost page write due to I/O error on md1
[27095.140697] Buffer I/O error on device md1, logical block 455870854
[27095.140714] lost page write due to I/O error on md1
[27095.141327] I/O error in filesystem ("md1") meta-data dev md1 block
0xaeaa9810 ("xlog_iodone") error 5 buf count 12288
[27095.141359] xfs_force_shutdown(md1,0x2) called from line 1056 of file
/build/buildd/linux-2.6.27/fs/xfs/xfs_log.c. Return address =
0xffffffffa04c80d3
[27095.141380] Filesystem "md1": Log I/O Error Detected. Shutting down
filesystem: md1
[27095.141407] Please umount the filesystem, and rectify the problem(s)
[27100.140015] Filesystem "md1": xfs_log_force: error 5 returned.
[27113.440011] Filesystem "md1": xfs_log_force: error 5 returned.
[27143.440010] Filesystem "md1": xfs_log_force: error 5 returned.
[27173.440009] Filesystem "md1": xfs_log_force: error 5 returned.
[27203.440012] Filesystem "md1": xfs_log_force: error 5 returned.
/var/log/messages:
Nov 30 18:39:24 isis kernel: [ 1061.040118] md: recovery of RAID array md1
Nov 30 18:39:24 isis kernel: [ 1061.040120] md: minimum _guaranteed_ speed:
1000 KB/sec/disk.
Nov 30 18:39:24 isis kernel: [ 1061.040122] md: using maximum available idle IO
bandwidth (but not more than 200000 KB/sec) for recovery.
Nov 30 18:39:24 isis kernel: [ 1061.040126] md: using 128k window, over a total
of 488383744 blocks.
Nov 30 19:02:08 isis -- MARK --
Nov 30 19:22:08 isis -- MARK --
Nov 30 19:42:08 isis -- MARK --
Nov 30 20:02:08 isis -- MARK --
Nov 30 20:22:08 isis -- MARK --
Nov 30 20:42:08 isis -- MARK --
Nov 30 21:02:08 isis -- MARK --
Nov 30 21:22:08 isis -- MARK --
Nov 30 21:28:32 isis kernel: [11208.852220] md: md1: recovery done.
Nov 30 21:28:32 isis kernel: [11209.020072] RAID5 conf printout:
Nov 30 21:28:32 isis kernel: [11209.020076] --- rd:7 wd:7
Nov 30 21:28:32 isis kernel: [11209.020079] disk 0, o:1, dev:sdd1
Nov 30 21:28:32 isis kernel: [11209.020080] disk 1, o:1, dev:sdb1
Nov 30 21:28:32 isis kernel: [11209.020081] disk 2, o:1, dev:sdh1
Nov 30 21:28:32 isis kernel: [11209.020082] disk 3, o:1, dev:sdc1
Nov 30 21:28:32 isis kernel: [11209.020083] disk 4, o:1, dev:sdf1
Nov 30 21:28:32 isis kernel: [11209.020084] disk 5, o:1, dev:sde1
Nov 30 21:28:32 isis kernel: [11209.020085] disk 6, o:1, dev:sdg1
Nov 30 21:42:08 isis -- MARK --
Nov 30 22:02:08 isis -- MARK --
Nov 30 22:22:08 isis -- MARK --
Nov 30 22:42:08 isis -- MARK --
Nov 30 23:02:08 isis -- MARK --
Nov 30 23:22:08 isis -- MARK --
Nov 30 23:42:08 isis -- MARK --
Nov 30 23:52:27 isis kernel: [19844.431690] SGI XFS with ACLs, security
attributes, realtime, large block/inode numbers, no debug enabled
Nov 30 23:52:27 isis kernel: [19844.433148] SGI XFS Quota Management subsystem
Nov 30 23:52:27 isis kernel: [19844.442507] Filesystem "md1": Disabling
barriers, trial barrier write failed
Nov 30 23:52:27 isis kernel: [19844.442658] XFS mounting filesystem md1
Dec 1 00:22:08 isis -- MARK --
Dec 1 00:42:08 isis -- MARK --
Dec 1 01:02:08 isis -- MARK --
Dec 1 01:22:08 isis -- MARK --
Dec 1 01:42:08 isis -- MARK --
Dec 1 01:52:10 isis kernel: [27027.170099] ata5: hard resetting link
Dec 1 01:52:10 isis kernel: [27027.680034] ata5: SATA link up 3.0 Gbps
(SStatus 123 SControl 300)
Dec 1 01:52:11 isis kernel: [27027.720050] ata5.00: max_sectors limited to 256
for NCQ
Dec 1 01:52:11 isis kernel: [27027.780047] ata5.00: max_sectors limited to 256
for NCQ
Dec 1 01:52:11 isis kernel: [27027.780050] ata5.00: configured for UDMA/133
Dec 1 01:52:11 isis kernel: [27027.780073] md: super_written gets error=-5,
uptodate=0
Dec 1 01:52:11 isis kernel: [27027.780117] ata5: EH complete
Dec 1 01:52:11 isis kernel: [27027.780674] sd 4:0:0:0: [sdg] 976773168
512-byte hardware sectors (500108 MB)
Dec 1 01:52:11 isis kernel: [27027.780800] sd 4:0:0:0: [sdg] Write Protect is
off
Dec 1 01:52:11 isis kernel: [27027.781038] sd 4:0:0:0: [sdg] Write cache:
enabled, read cache: enabled, doesn't support DPO or FUA
Dec 1 01:52:41 isis kernel: [27057.930098] ata12: hard resetting link
Dec 1 01:52:41 isis kernel: [27058.440033] ata12: SATA link up 3.0 Gbps
(SStatus 123 SControl 300)
Dec 1 01:52:41 isis kernel: [27058.480049] ata12.00: max_sectors limited to
256 for NCQ
Dec 1 01:52:41 isis kernel: [27058.540047] ata12.00: max_sectors limited to
256 for NCQ
Dec 1 01:52:41 isis kernel: [27058.540050] ata12.00: configured for UDMA/133
Dec 1 01:52:41 isis kernel: [27058.540072] md: super_written gets error=-5,
uptodate=0
Dec 1 01:52:41 isis kernel: [27058.540113] ata12: EH complete
Dec 1 01:52:41 isis kernel: [27058.540754] sd 11:0:0:0: [sdh] 976773168
512-byte hardware sectors (500108 MB)
Dec 1 01:52:41 isis kernel: [27058.540879] sd 11:0:0:0: [sdh] Write Protect is
off
Dec 1 01:52:41 isis kernel: [27058.541070] sd 11:0:0:0: [sdh] Write cache:
enabled, read cache: enabled, doesn't support DPO or FUA
Dec 1 01:52:41 isis kernel: [27058.584017] RAID5 conf printout:
Dec 1 01:52:41 isis kernel: [27058.584020] --- rd:7 wd:5
Dec 1 01:52:41 isis kernel: [27058.584022] disk 0, o:1, dev:sdd1
Dec 1 01:52:41 isis kernel: [27058.584023] disk 1, o:1, dev:sdb1
Dec 1 01:52:41 isis kernel: [27058.584024] disk 2, o:0, dev:sdh1
Dec 1 01:52:41 isis kernel: [27058.584025] disk 3, o:1, dev:sdc1
Dec 1 01:52:41 isis kernel: [27058.584027] disk 4, o:1, dev:sdf1
Dec 1 01:52:41 isis kernel: [27058.584028] disk 5, o:1, dev:sde1
Dec 1 01:52:41 isis kernel: [27058.584029] disk 6, o:0, dev:sdg1
Dec 1 01:52:44 isis kernel: [27061.521251] Modules linked in: xfs aes_x86_64
aes_generic ecb crypto_blkcipher ecryptfs ipv6 af_packet iptable_filter
ip_tables x_tables ac sbp2 parport_pc lp parport loop psmouse pcspkr serio_raw
iTCO_wdt iTCO_vendor_support evdev button intel_agp snd_hda_intel snd_pcm
shpchp snd_timer pci_hotplug snd soundcore snd_page_alloc ext3 jbd mbcache
sd_mod crc_t10dif sg pata_acpi pata_marvell usbhid hid ohci1394 ieee1394
sata_mv ata_generic ata_piix libata scsi_mod dock sky2 e1000e ehci_hcd uhci_hcd
usbcore raid10 raid456 async_xor async_memcpy async_tx xor raid1 raid0
multipath linear md_mod dm_mirror dm_log dm_snapshot dm_mod thermal processor
fan fbcon tileblit font bitblit softcursor fuse
Dec 1 01:52:44 isis kernel: [27061.521251] CPU 1:
Dec 1 01:52:44 isis kernel: [27061.521251] Modules linked in: xfs aes_x86_64
aes_generic ecb crypto_blkcipher ecryptfs ipv6 af_packet iptable_filter
ip_tables x_tables ac sbp2 parport_pc lp parport loop psmouse pcspkr serio_raw
iTCO_wdt iTCO_vendor_support evdev button intel_agp snd_hda_intel snd_pcm
shpchp snd_timer pci_hotplug snd soundcore snd_page_alloc ext3 jbd mbcache
sd_mod crc_t10dif sg pata_acpi pata_marvell usbhid hid ohci1394 ieee1394
sata_mv ata_generic ata_piix libata scsi_mod dock sky2 e1000e ehci_hcd uhci_hcd
usbcore raid10 raid456 async_xor async_memcpy async_tx xor raid1 raid0
multipath linear md_mod dm_mirror dm_log dm_snapshot dm_mod thermal processor
fan fbcon tileblit font bitblit softcursor fuse
Dec 1 01:52:44 isis kernel: [27061.521251] Pid: 28171, comm: smbd Not tainted
2.6.27-9-server #1
Dec 1 01:52:44 isis kernel: [27061.521251] RIP: 0010:[<ffffffff802abf0c>]
[<ffffffff802abf0c>] find_get_pages+0x6c/0x110
Dec 1 01:52:44 isis kernel: [27061.521251] RSP: 0018:ffff880129453358 EFLAGS:
00000246
Dec 1 01:52:44 isis kernel: [27061.521251] RAX: ffff880128d89330 RBX:
ffff880129453398 RCX: 0000000000000002
Dec 1 01:52:44 isis kernel: [27061.521251] RDX: 0000000000000003 RSI:
0000000000000000 RDI: ffffe200022e9e80
Dec 1 01:52:44 isis kernel: [27061.521251] RBP: ffff880129453308 R08:
ffffe200009df6c8 R09: 0000000000000005
Dec 1 01:52:44 isis kernel: [27061.521251] R10: 0000000000000037 R11:
00000000001c5778 R12: ffffffff802b6b29
Dec 1 01:52:44 isis kernel: [27061.521251] R13: ffff880123a107d0 R14:
ffffe20001c6f6c0 R15: 0000000000000286
Dec 1 01:52:44 isis kernel: [27061.521251] FS: 00007fb72cdf6700(0000)
GS:ffff88012fc02980(0000) knlGS:0000000000000000
Dec 1 01:52:44 isis kernel: [27061.521251] CS: 0010 DS: 0000 ES: 0000 CR0:
000000008005003b
Dec 1 01:52:44 isis kernel: [27061.521251] CR2: 00007f1648629000 CR3:
000000012956d000 CR4: 00000000000006e0
Dec 1 01:52:44 isis kernel: [27061.521251] DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Dec 1 01:52:44 isis kernel: [27061.521251] DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Dec 1 01:52:44 isis kernel: [27061.521251]
Dec 1 01:52:44 isis kernel: [27061.521251] Call Trace:
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802abee3>] ?
find_get_pages+0x43/0x110
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b6984>] ?
pagevec_lookup+0x24/0x30
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e100d>] ?
xfs_cluster_write+0xad/0x180 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e1578>] ?
xfs_page_state_convert+0x498/0x760 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e19a1>] ?
xfs_vm_writepage+0x71/0x120 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b9274>] ?
pageout+0x124/0x270
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ab06a>] ?
page_waitqueue+0xa/0x90
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b986d>] ?
shrink_page_list+0x34d/0x530
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b8e49>] ?
__isolate_lru_page+0x79/0xb0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b8f0a>] ?
isolate_lru_pages+0x8a/0x220
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b9bf2>] ?
shrink_inactive_list+0x1a2/0x4b0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b9f7b>] ?
shrink_zone+0x7b/0x160
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ba0ed>] ?
shrink_zones+0x8d/0x150
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ba236>] ?
do_try_to_free_pages+0x86/0x2e0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ba587>] ?
try_to_free_pages+0x67/0x70
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b90a0>] ?
isolate_pages_global+0x0/0x50
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802b28b1>] ?
__alloc_pages_internal+0x241/0x510
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802d565d>] ?
alloc_pages_current+0xad/0x110
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ac477>] ?
__page_cache_alloc+0x67/0x80
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ad0b3>] ?
__grab_cache_page+0x63/0xb0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff80316a59>] ?
block_write_begin+0x89/0xf0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e04ca>] ?
xfs_vm_write_begin+0x2a/0x30 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e0040>] ?
xfs_get_blocks+0x0/0x20 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ab7ac>] ?
generic_perform_write+0xbc/0x1c0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802ad512>] ?
generic_file_buffered_write+0x92/0x170
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e92d3>] ?
xfs_write+0x6b3/0x9b0 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff80385a69>] ?
apparmor_socket_recvmsg+0x19/0x20
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff803aaf70>] ?
memset_c+0x20/0x30
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffffa04e4c88>] ?
xfs_file_aio_write+0x58/0x60 [xfs]
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802e9559>] ?
do_sync_write+0xf9/0x140
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802e9699>] ?
do_sync_read+0xf9/0x140
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff80266fb0>] ?
autoremove_wake_function+0x0/0x40
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff80386821>] ?
aa_file_permission+0x21/0xf0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff80386948>] ?
apparmor_file_permission+0x28/0x30
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff803613e6>] ?
security_file_permission+0x16/0x20
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802e9c1b>] ?
vfs_write+0xcb/0x130
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff802e9d1a>] ?
sys_pwrite64+0x9a/0xa0
Dec 1 01:52:44 isis kernel: [27061.521251] [<ffffffff8021285a>] ?
system_call_fastpath+0x16/0x1b
Dec 1 01:52:44 isis kernel: [27061.521251]
Dec 1 01:53:18 isis kernel: [27095.080066] RAID5 conf printout:
Dec 1 01:53:18 isis kernel: [27095.080071] --- rd:7 wd:5
Dec 1 01:53:18 isis kernel: [27095.080074] disk 0, o:1, dev:sdd1
Dec 1 01:53:18 isis kernel: [27095.080076] disk 1, o:1, dev:sdb1
Dec 1 01:53:18 isis kernel: [27095.080077] disk 2, o:0, dev:sdh1
Dec 1 01:53:18 isis kernel: [27095.080079] disk 3, o:1, dev:sdc1
Dec 1 01:53:18 isis kernel: [27095.080080] disk 4, o:1, dev:sdf1
Dec 1 01:53:18 isis kernel: [27095.080082] disk 5, o:1, dev:sde1
Dec 1 01:53:18 isis kernel: [27095.080090] RAID5 conf printout:
Dec 1 01:53:18 isis kernel: [27095.080091] --- rd:7 wd:5
Dec 1 01:53:18 isis kernel: [27095.080092] disk 0, o:1, dev:sdd1
Dec 1 01:53:18 isis kernel: [27095.080093] disk 1, o:1, dev:sdb1
Dec 1 01:53:18 isis kernel: [27095.080094] disk 2, o:0, dev:sdh1
Dec 1 01:53:18 isis kernel: [27095.080095] disk 3, o:1, dev:sdc1
Dec 1 01:53:18 isis kernel: [27095.080097] disk 4, o:1, dev:sdf1
Dec 1 01:53:18 isis kernel: [27095.080098] disk 5, o:1, dev:sde1
Dec 1 01:53:18 isis kernel: [27095.140011] RAID5 conf printout:
Dec 1 01:53:18 isis kernel: [27095.140017] --- rd:7 wd:5
Dec 1 01:53:18 isis kernel: [27095.140019] disk 0, o:1, dev:sdd1
Dec 1 01:53:18 isis kernel: [27095.140022] disk 1, o:1, dev:sdb1
Dec 1 01:53:18 isis kernel: [27095.140024] disk 3, o:1, dev:sdc1
Dec 1 01:53:18 isis kernel: [27095.140026] disk 4, o:1, dev:sdf1
Dec 1 01:53:18 isis kernel: [27095.140027] disk 5, o:1, dev:sde1
Dec 1 01:53:18 isis kernel: [27095.140545] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140567] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140585] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140604] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140622] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140641] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140659] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140677] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140696] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.140714] lost page write due to I/O error on
md1
Dec 1 01:53:18 isis kernel: [27095.141359] xfs_force_shutdown(md1,0x2) called
from line 1056 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_log.c. Return
address = 0xffffffffa04c80d3
Dec 1 01:53:23 isis kernel: [27100.140015] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:53:36 isis kernel: [27113.440011] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:54:06 isis kernel: [27143.440010] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:54:36 isis kernel: [27173.440009] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:55:06 isis kernel: [27203.440012] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:55:36 isis kernel: [27233.440011] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:56:06 isis kernel: [27263.440011] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:56:36 isis kernel: [27293.440010] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:57:06 isis kernel: [27323.440016] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:57:36 isis kernel: [27353.440015] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:58:06 isis kernel: [27383.440015] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 01:58:36 isis kernel: [27413.440016] Filesystem "md1": xfs_log_force:
error 5 returned.
^^^^^^continues this for a while
Dec 1 02:12:06 isis kernel: [28223.440015] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:12:36 isis kernel: [28253.440013] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:13:06 isis kernel: [28283.440014] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:13:36 isis kernel: [28313.440013] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:06 isis kernel: [28343.440013] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:36 isis kernel: [28373.440012] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28395.820448] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28395.820456] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28395.820462] xfs_force_shutdown(md1,0x1) called
from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c. Return
address = 0xffffffffa04decc3
Dec 1 02:14:59 isis kernel: [28395.820466] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28395.820468] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28395.820471] xfs_force_shutdown(md1,0x1) called
from line 420 of file /build/buildd/linux-2.6.27/fs/xfs/xfs_rw.c. Return
address = 0xffffffffa04decc3
Dec 1 02:14:59 isis kernel: [28396.669470] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28396.669487] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28396.669517] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28396.669525] Filesystem "md1": xfs_log_force:
error 5 returned.
Dec 1 02:14:59 isis kernel: [28396.669635] Filesystem "md1": xfs_log_force:
error 5 returned.
--
hardy / ibex - raid5 - ata#: hard resetting link
https://bugs.launchpad.net/bugs/263160
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
