Hi all,
I recently experienced a disk failure on my home server and observed checksum
errors while resilvering the pool and on the first scrub after the resilver had
completed. Now everything seems fine but I'm posting this to get help with
calming my nerves and detect any possible future faults.
Lets start with some specs.
OSOL 2009.06
Intel SASUC8i (w LSI 1.30IT FW)
Gigabyte MA770-UD3 mobo w 8GB ECC RAM
Hitachi P7K500 harddrives
When checking the condition of my pool some days ago (yes I should make it
mail me if something like this happens again) one disk in my pool was labeled
as "Removed" with a small number of read errors, nineish I think, all other
disks where fine. I removed tested (DFT crashed so the disk seemed very broken)
replaced the drive and started a resilver.
Checking the status of the resilver everything looked good from the start but
when it was finished the status report looked like this:
pool: sasuc8i
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://www.sun.com/msg/ZFS-8000-9P
scrub: resilver completed after 4h9m with 0 errors on Mon Apr 12 18:12:26 2010
config:
NAME STATE READ WRITE CKSUM
sasuc8i ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c12t4d0 ONLINE 0 0 5 108K resilvered
c12t8d0 ONLINE 0 0 0 254G resilvered
c12t6d0 ONLINE 0 0 0
c12t7d0 ONLINE 0 0 0
c12t0d0 ONLINE 0 0 1 21.5K resilvered
c12t1d0 ONLINE 0 0 2 43K resilvered
c12t2d0 ONLINE 0 0 4 86K resilvered
c12t3d0 ONLINE 0 0 1 21.5K resilvered
errors: No known data errors
All I really cared about at this point was the "Applications are unaffected"
and "No known data errors" and I thought that the checksum errors might be down
to the failing drive (c12t5d0 failed, the controlled labeled the new drive as
c12t8d0) going out during a write. Then again ZFS is atomic, better clear the
errors and run a scrub, it came out like this:
pool: sasuc8i
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://www.sun.com/msg/ZFS-8000-9P
scrub: scrub completed after 1h16m with 0 errors on Tue Apr 13 01:29:32 2010
config:
NAME STATE READ WRITE CKSUM
sasuc8i ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c12t4d0 ONLINE 0 0 5
c12t8d0 ONLINE 0 0 0
c12t6d0 ONLINE 0 0 0
c12t7d0 ONLINE 0 0 4 86K repaired
c12t0d0 ONLINE 0 0 1
c12t1d0 ONLINE 0 0 6 86K repaired
c12t2d0 ONLINE 0 0 4
c12t3d0 ONLINE 0 0 6 108K repaired
errors: No known data errors
Now I'm getting nervous. Checksum errors, some repaired others not. Am I going
to end up with multiple drive failures or what the * is going on here?
Ran one more scrub and everything came up roses.
Checked smart status on the drives with checksum errors and they are fine,
allthough I expect only read/write errors would show up there.
I'm not sure of how to get this into a propper question but what I'm after is
"is this normal to be expected after a resilver and can I start breathing
again?". Checksum errors are as far as I can gather dodgy data on disk and
read/write somewhere in the physical link (more or less).
Thank you!
--
This message posted from opensolaris.org
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
