Re: [zfs-discuss] Scrub and checksum permutations

2012-10-31 Thread Matthew Ahrens
On Thu, Oct 25, 2012 at 2:25 AM, Jim Klimov jimkli...@cos.ru wrote: Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror halves? Yes. The ZIO_FLAG_SCRUB instructs the raidz or mirror vdev to read

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-31 Thread Timothy Coalson
On Wed, Oct 31, 2012 at 6:47 PM, Matthew Ahrens mahr...@delphix.com wrote: On Thu, Oct 25, 2012 at 2:25 AM, Jim Klimov jimkli...@cos.ru wrote: Hello all, I was describing how raidzN works recently, and got myself wondering: does zpool scrub verify all the parity sectors and the mirror

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-29 Thread Tomas Forsman
On 28 October, 2012 - Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) sent me these 1,0K bytes: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov I tend to agree that parity calculations likely are faster (even if

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-28 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov I tend to agree that parity calculations likely are faster (even if not all parities are simple XORs - that would be silly for double- or triple-parity sets which may use

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Ray Arachelian
On 10/26/2012 04:29 AM, Karl Wagner wrote: Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. I could understand why it would not store a checksum for a

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Toby Thain
On 27/10/12 11:56 AM, Ray Arachelian wrote: On 10/26/2012 04:29 AM, Karl Wagner wrote: Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. ... Parity is

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Jim Klimov
2012-10-27 20:54, Toby Thain wrote: Parity is very simple to calculate and doesn't use a lot of CPU - just slightly more work than reading all the blocks: read all the stripe blocks on all the drives involved in a stripe, then do a simple XOR operation across all the data. The actual checksums

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-27 Thread Timothy Coalson
On Sat, Oct 27, 2012 at 12:35 PM, Jim Klimov jimkli...@cos.ru wrote: 2012-10-27 20:54, Toby Thain wrote: Parity is very simple to calculate and doesn't use a lot of CPU - just slightly more work than reading all the blocks: read all the stripe blocks on all the drives involved in a stripe,

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-26 Thread Karl Wagner
Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. I could understand why it would not store a checksum for a parity block. It is not really necessary:

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-26 Thread Jim Klimov
2012-10-26 12:29, Karl Wagner wrote: Does it not store a separate checksum for a parity block? If so, it should not even need to recalculate the parity: assuming checksums match for all data and parity blocks, the data is good. No, for the on-disk sector allocation over M disks, zfs raidzN

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Karl Wagner
I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to verify all redundant data. The point of a scrub is to ensure all data is correct. On 2012-10-25 10:25, Jim Klimov

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Karl Wagner I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on both disks in a mirrored pair. Logically, it would not make sense not to

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Jim Klimov
2012-10-25 15:30, Edward Ned Harvey (opensolarisisdeadlongliveopensolaris) пишет: From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Karl Wagner I can only speak anecdotally, but I believe it does. Watching zpool iostat it does read all data on

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jim Klimov Logically, yes - I agree this is what we expect to be done. However, at least with the normal ZFS reading pipeline, reads of redundant copies and parities only kick in if the

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Timothy Coalson
On Thu, Oct 25, 2012 at 7:35 AM, Jim Klimov jimkli...@cos.ru wrote: If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in order to ensure that parity sectors are intact and can be used for data

Re: [zfs-discuss] Scrub and checksum permutations

2012-10-25 Thread Jim Klimov
2012-10-25 21:17, Timothy Coalson wrote: On Thu, Oct 25, 2012 at 7:35 AM, Jim Klimov jimkli...@cos.ru mailto:jimkli...@cos.ru wrote: If scrubbing works the way we logically expect it to, it should enforce validation of such combinations for each read of each copy of a block, in