2012-05-22 7:30, Daniel Carosone wrote:
On Mon, May 21, 2012 at 09:18:03PM -0500, Bob Friesenhahn wrote:
On Mon, 21 May 2012, Jim Klimov wrote:
This is so far a relatively raw idea and I've probably missed
something. Do you think it is worth pursuing and asking some
zfs developers to make a POC? ;)

I did read all of your text. :-)

This is an interesting idea and could be of some use but it would be
wise to test it first a few times before suggesting it as a general

I've done basically this kind of thing before: dd a disk and then
scrub rather than replace, treating errors as expected.

I got into similar situation last night on that Thumper -
it is now migrating a flaky source disk in the array from
an original old 250Gb disk into a same-sized partition on
the new 3Tb drive (as I outlined as IDEA7 in another thread).
The source disk itself had about 300 CKSUM errors during
the process, and for reasons beyond my current understanding,
the resilver never completed.

In zpool status it said that the process was done several
hours before the time I looked at it, but the TLVDEV still
had a "spare" component device comprised of the old disk
and new partition, and the (same) hotspare device in the
pool was "INUSE".

After a while we just detached the old disk from the pool
and ran scrub, which first found some 178 CKSUM errors on
the new partition right away, and degraded the TLVDEV and

We cleared the errors, and ran the script below to log
the detected errors and clear them, so the disk is fixed
and not kicked out of the pool due to mismatches.
Overall 1277 errors were logged and apparently fixed, and
the pool is now on its second full scrub run - no bugs so
far (knocking wood; certainly none this early in the scrub
as we had last time).

So in effect, this methodology works for two of us :)

Since you did similar stuff already, I have a few questions:
1) How/what did you DD? The whole slice with the zfs vdev?
   Did the system complain (much) about the renaming of the
   device compared to paths embedded in pool/vdev headers?
   Did you do anything manually to remedy that (forcing
   import, DDing some handcrafted uberblocks, anything?)

2) How did you "treat errors as expected" during scrub?
   As I've discovered, there were hoops to jump through.
   Is there a switch to disable "degrading" of pools and
   TLVDEVs based on only the CKSUM counts?

My raw hoop-jumping script:


# /root/scrubwatch.sh
# Watches 'pond' scrub and resets errors to avoid auto-degrading
# the device, but logs the detected error counts however.
# See also "fmstat|grep zfs-diag" for precise counts.
# See also https://blogs.oracle.com/bobn/entry/zfs_and_fma_two_great
#          for details on FMA and fmstat with zfs hotspares

while true; do
    zpool status pond | gegrep -A4 -B3 'resilv|error|c1t2d|c5t6d|%'
    echo ""

    C1="`zpool status pond | grep c1t2d`"
    C2="`echo "$C1" | grep 'c1t2d0s1  ONLINE       0     0     0'`"
    if [ x"$C2" = x ]; then
        echo "`date`: $C1" >> /var/tmp/zpool-clear_pond.log
        zpool clear pond
        zpool status pond | gegrep -A4 -B3 'resilv|error|c1t2d|c5t6d|%'
    echo ""

    sleep 60

//Jim Klimov
zfs-discuss mailing list

Reply via email to