Hi all,
I'm testing my first net6501 with an OCZ Petrol SSD. Everything seemed OK, got
the OS installed, then I started getting checksum mismatches from rsync, then
I/O errors, and then the drive stopped working completely.
The rsync errors started like this:
+ rsync -avP --delete --exclude /dev --exclude /proc --exclude /sys --exclude
/tmp --exclude /var/tmp --exclude /var/log --exclude /var/run --exclude
/var/lib/postfix 10.0.156.222::root/ /mnt/target/
media/hdb1/system-imager/image/usr/lib/locale/en_CA.utf8/LC_TIMEWARNING:
media/hdb1/system-imager/image/usr/lib/locale/en_IE.utf8/LC_CTYPE failed
verification -- update discarded (will try again).
WARNING: media/hdb1/system-imager/image/usr/lib/locale/en_NZ.utf8/LC_CTYPE
failed verification -- update discarded (will try again).
WARNING: media/hdb1/system-imager/image/usr/lib/locale/en_PH.utf8/LC_CTYPE
failed verification -- update discarded (will try again).
Nothing in dmesg at this point. Then it went like this:
rsync:
readlink_stat("/mnt/target/media/hdb1/system-imager/image/usr/share/icons/Humanity/mimes/16/application-vnd.ms-powerpoint.svg")
failed: Input/output error (5)
rsync: recv_generator: failed to stat
"/mnt/target/media/hdb1/system-imager/image/usr/share/icons/Humanity/mimes/16/application-vnd.ms-powerpoint.svg":
Input/output error (5)
rsync: symlink
"/mnt/target/media/hdb1/system-imager/image/usr/share/icons/Humanity/mimes/48/application-x-tar.svg"
-> "tar.svg" failed: Input/output error (5)
I checked the drive for hardware errors using smartctl, and it status seemed
good:
root@system-imager:~# smartctl -a /dev/sda
smartctl 5.40 2010-07-12 r3124 [i686-pc-linux-gnu] (local build)
Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net
=== START OF INFORMATION SECTION ===
Device Model: OCZ-PETROL
Serial Number: OCZ-25G7LV2577Z5OGG8
Firmware Version: 3.12
User Capacity: 128,035,676,160 bytes
Device is: Not in smartctl database [for details use: -P showall]
ATA Version is: 8
ATA Standard is: Exact ATA specification draft version not indicated
Local Time is: Wed Jan 2 05:47:24 1980 GMT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x00) Offline data collection activity
was never started.
Auto Offline Data Collection: Disabled.
Self-test execution status: ( 249) Self-test routine in progress...
90% of test remaining.
Total time to complete Offline
data collection: ( 0) seconds.
Offline data collection
capabilities: (0x1d) SMART execute Offline immediate.
No Auto Offline data collection
support.
Abort Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
No Conveyance Self-test supported.
No Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x00) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 0) minutes.
Extended self-test routine
recommended polling time: ( 0) minutes.
SMART Attributes Data Structure revision number: 18
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x0000 006 000 000 Old_age Offline -
6
3 Spin_Up_Time 0x0000 100 100 000 Old_age Offline -
0
4 Start_Stop_Count 0x0000 100 100 000 Old_age Offline -
0
5 Reallocated_Sector_Ct 0x0000 100 100 000 Old_age Offline -
0
9 Power_On_Hours 0x0000 100 100 000 Old_age Offline -
5
12 Power_Cycle_Count 0x0000 100 100 000 Old_age Offline -
8
232 Available_Reservd_Space 0x0000 100 100 000 Old_age Offline -
160
233 Media_Wearout_Indicator 0x0000 100 000 000 Old_age Offline -
100
SMART Error Log Version: 0
No Errors Logged
So I thought "filesystem corruption", and tried to unmount the drive for a
fsck, which took forever, and then I started seeing errors in the system logs:
[15179.138794] ata1.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen
[15179.142753] ata1.00: failed command: WRITE FPDMA QUEUED
[15179.146771] ata1.00: cmd 61/08:00:d5:29:c4/00:00:07:00:00/40 tag 0 ncq 4096
out
[15179.146780] res 40/00:00:06:4f:c2/00:00:00:00:00/00 Emask 0x4
(timeout)
[15179.150745] ata1.00: status: { DRDY }
[15179.154761] ata1: hard resetting link
[15189.171385] ata1: softreset failed (device not ready)
[15189.175375] ata1: hard resetting link
[15199.192010] ata1: softreset failed (device not ready)
[15199.196001] ata1: hard resetting link
[15209.772672] ata1: link is slow to respond, please be patient (ready=0)
[15234.250202] ata1: softreset failed (device not ready)
[15234.254196] ata1: limiting SATA link speed to 1.5 Gbps
[15234.258190] ata1: hard resetting link
[15239.450526] ata1: softreset failed (device not ready)
[15239.454515] ata1: reset failed, giving up
[15239.458515] ata1.00: disabled
[15239.462551] ata1: EH complete
[15239.466657] sd 0:0:0:0: [sda] Unhandled error code
[15239.470518] sd 0:0:0:0: [sda] Result: hostbyte=DID_BAD_TARGET
driverbyte=DRIVER_OK
[15239.478514] sd 0:0:0:0: [sda] CDB: Write(10): 2a 00 07 c4 29 d5 00 00 08 00
[15239.503544] end_request: I/O error, dev sda, sector 130296277
[15239.506522] Buffer I/O error on device sda2, logical block 14188544
[15239.510516] lost page write due to I/O error on sda2
[15239.514581] JBD2: I/O error detected when updating journal superblock for
sda2-8.
Then I couldn't read from any part of the device, or run smartctl on it any
more. Badblocks shows every block as bad, and I get errors in dmesg about them
too. I guess this is because Linux gave up trying to reset the drive, so the
hardware (drive or AHCI or both) is stuck in a bad state and communication is
impossible or corrupted.
I've seen this link, and I know Linus Torvalds would say that my disk is bad,
despite the good name of OCZ and the good smartctl report:
https://plus.google.com/113641248237520845183/posts/SXWeXVY7Y6p
But after a power off (not just a hard reset) the drive is fine again!
Some people are seeing issues with Debian kernels (like this one) and multiple
drives appearing to fail at the same time, but working fine after a reboot:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=625922
So I suspect there's nothing wrong with the drive; it may be a firmware bug,
but I was unable to apply firmware fixes using OCZ's updater for Linux. Has
anyone seen anything like this? Anyone got a sense for whether it could be a
hardware bug with the AHCI controller or BIOS on the net6501?
Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK
Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.
_______________________________________________
Soekris-tech mailing list
Soekris-tech@lists.soekris.com
http://lists.soekris.com/mailman/listinfo/soekris-tech
