more below...

On 2012-12-05 23:16, Timothy Coalson wrote:
On Tue, Dec 4, 2012 at 10:52 PM, Jim Klimov <
<>> wrote:

    On 2012-12-03 18:23, Jim Klimov wrote:

        On 2012-12-02 05:42, Jim Klimov wrote:

     >> 4) Where are the redundancy algorithms specified? Is there any
     >>     tool that would recombine a given algo-N redundancy sector with
     >>     some other 4 sectors from a 6-sector stripe in order to try and
     >>     recalculate the sixth sector's contents? (Perhaps part of some
     >>     unit tests?)

I'm a bit late to the party, but from a previous list thread about
redundancy algorithms, I had found this:

Particularly the
functions "vdev_raidz_reconstruct_p", "vdev_raidz_reconstruct_q", 
(and possibly "vdev_raidz_reconstruct_general") seem like what you are
looking for.

As I understand it, the case where you have both redundancy blocks, but
are missing two data blocks, is the hardest (if you are missing only one
data block, you can either do straight xor with the first redundancy
section, or some LFSR shifting, xor, and then reverse LFSR shifting to
use the second redundancy section).

Wikipedia describes the math to restore from two missing data sections
here, under "computing parity":

I don't know any tools to do this for you from arbitrary input, sorry.

Thanks, you are not "late" and welcome to the party ;)

I'm hacking together a simple program to look over the data sectors and
XOR parity and determine how many, if any, discrepancies there are, and
at what offsets into the sector - byte by byte.

Running it on raw ZFS block component sectors, extracted with DD in the
ways I wrote of earlier in the thread, I did confirm some good sectors
and the one erroneous block that I have.

The latter turns out to have 4.5 worth of sectors in userdata, overall
laid out like this:
   dsk0  dsk1  dsk2  dsk3  dsk4  dsk5
   _     _     _     _     _     p1
   q1    d0    d2    d3    d4*   p2
   q2    d1    _     _     _     _

Here the compressed userdata is contained in the order of my "d"-sector
numbering, d0-d1-d2-d3-d4, and d4 is only partially occupied (P-size of
the block is 0x4c00) so its final quarter is all zeroes.

It also happens that on disks 1,2,3 the first row's sectors (d0, d2, d3)
are botched - ranges from 0x9C0 to 0xFFF (end of 4KB sector) are zeroes.

The neighboring blocks, located a few sectors away from this one, also
have compressed data and have some regular-looking patterns of bytes,
certainly no long stretches of zeroes.

However, the byte-by-byte XOR matching complains about the whole sector.
All bytes, except some 40 single-byte locations here and there, don't
XOR up to produce the expected (known from disk) value.

I did not yet try the second parity algorithm.

At least in this case, it does not seem that I would find an incantation
needed to recover this block - too many zeroes overlapping (at least 3
disks' data proven compromised), where I did hope for some shortcoming
in ZFS recombination exhaustiveness. In this case - it is indeed too
much failure to handle.

Now waiting for scrub to find me more test subjects - broken files ;)

zfs-discuss mailing list

Reply via email to