On 2011-04-11 10:28, Roy Sigurd Karlsbakk wrote:
Hi all

I have this box for backup storage, urd-backup, zfs receiving a copy of another 
system, urd. on urd-backup, I created the following pool

zpool create -f urd-backup \
         raidz2 c5t0d0 c5t1d0 c5t2d0 c5t3d0 c5t4d0 c5t5d0 c5t6d0 c5t7d0 c5t8d0 \
         raidz2 c5t9d0 c5t10d0 c5t11d0 c5t12d0 c5t13d0 c5t14d0 c5t15d0 c5t16d0 
c5t17d0 \
         raidz2 c5t18d0 c5t19d0 c5t20d0 c5t21d0 c5t22d0 c5t23d0 c5t24d0 c5t25d0 
c5t26d0 \
         raidz2 c5t27d0 c5t28d0 c5t29d0 c5t30d0 c5t31d0 c5t32d0 c5t33d0 c5t34d0 
c5t35d0

Something recently happened, and the pool was suddenly showing lots of bad 
drives. Also, it now shows the drives in different order than the original, 
which I find rather alarming. I also get an error message when trying to zpool 
replace a drive to attempt a forced resilver of that. See below the zpool 
status output for this.

Does anyone know what on earth could have caused this, and how I can fix it?

root@urd-backup:~# zpool status urd-backup
   pool: urd-backup
  state: DEGRADED
status: One or more devices are faulted in response to persistent errors.
         Sufficient replicas exist for the pool to continue functioning in a
         degraded state.
action: Replace the faulted device, or use 'zpool clear' to mark the device
         repaired.
  scan: scrub in progress since Mon Apr 11 10:25:44 2011
     14.4M scanned out of 37.7T at 14.4M/s, (scan is slow, no estimated time)
     0 repaired, 0.00% done
config:

         NAME         STATE     READ WRITE CKSUM
         urd-backup   DEGRADED     0     0     0
           raidz2-0   DEGRADED     0     0     0
             c5t14d0  ONLINE       0     0     0
             c5t16d0  DEGRADED     0     0     0  too many errors
             c5t18d0  ONLINE       0     0     0
             c5t23d0  DEGRADED     0     0     0  too many errors
             c5t27d0  ONLINE       0     0     0
             c5t5d0   FAULTED      0     0     0  corrupted data
             c5t32d0  ONLINE       0     0     0
             c5t20d0  DEGRADED     0     0     0  too many errors
             c5t26d0  ONLINE       0     0     0
           raidz2-1   ONLINE       0     0     0
             c5t30d0  ONLINE       0     0     0
             c5t31d0  ONLINE       0     0     0
             c5t12d0  ONLINE       0     0     0
             c5t13d0  ONLINE       0     0     0
             c5t0d0   ONLINE       0     0     0
             c5t1d0   ONLINE       0     0     0
             c5t2d0   ONLINE       0     0     0
             c5t3d0   ONLINE       0     0     0
             c5t4d0   ONLINE       0     0     0
           raidz2-2   DEGRADED     0     0     0
             c5t5d0   ONLINE       0     0     0
             c5t6d0   ONLINE       0     0     0
             c5t7d0   ONLINE       0     0     0
             c5t8d0   ONLINE       0     0     0
             c5t9d0   ONLINE       0     0     0
             c5t10d0  ONLINE       0     0     0
             c5t11d0  ONLINE       0     0     0
             c5t15d0  ONLINE       0     0     0
             c5t26d0  FAULTED      0     0     0  too many errors
           raidz2-3   DEGRADED     0     0     0
             c5t19d0  ONLINE       0     0     0
             c5t21d0  ONLINE       0     0     0
             c5t22d0  ONLINE       0     0     0
             c5t24d0  ONLINE       0     0     0
             c5t31d0  FAULTED      0     0     0  corrupted data
             c5t25d0  ONLINE       0     0     0
             c5t33d0  FAULTED      0     0     0  too many errors
             c5t28d0  ONLINE       0     0     0
             c5t29d0  DEGRADED     0     0     0  too many errors

errors: No known data errors
root@urd-backup:~# zpool replace -f urd-backup c5t5d0 c5t5d0
invalid vdev specification
the following errors must be manually repaired:
/dev/dsk/c5t5d0s0 is part of active ZFS pool urd-backup. Please see zpool(1M).
root@urd-backup:~#


It sounds like a bit too much of coincidence that 8 drives have failed. Maybe a port extender has failed. It would be of interest to know what these 8 drives have in common.

_______________________________________________
OpenIndiana-discuss mailing list
OpenIndiana-discuss@openindiana.org
http://openindiana.org/mailman/listinfo/openindiana-discuss

Reply via email to