> From: Jim Klimov [mailto:jimkli...@cos.ru]
> Sent: Thursday, July 12, 2012 8:42 AM
> To: Edward Ned Harvey
> Subject: Re: [zfs-discuss] New fast hash algorithm - is it needed?
> 2012-07-11 18:03, Edward Ned Harvey пишет:
> >> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
> >> boun...@opensolaris.org] On Behalf Of Sašo Kiselkov
> >> As your dedup
> >> ratio grows, so does the performance hit from dedup=verify. At, say,
> >> dedupratio=10.0x, on average, every write results in 10 reads.
> > Why?
> > If you intend to write a block, and you discover it has a matching checksum,
> > you only need to verify it matches one block. You don't need to verify it
> > matches all the other blocks that have previously been verified to match
> > each other.
> As Saso explained, if you wrote the same block 10 times
> and detected it was already deduped, then by verifying
> this detection 10 times you get about 10 extra reads.
(In this case, Jim, you wrote me off-list and I replied on-list, but in this
case, I thought it would be ok because this message doesn't look private or
exclusive to me. I apologize if I was wrong.)
I get the miscommunication now -
When you write the duplicate block for the 10th time, we all understand you're
not going to go back and verify 10 blocks. (It seemed, at least to me, that's
what Saso was saying. Which is why I told him, No you don't.)
You're saying, that when you wrote the duplicate block the 2nd time, you
verified... And then when you wrote it the 3rd time, you verified... And the
4th time, and the 5th time... By the time you write the 10th time, you have
already verified 9 previous times, but you're still going to verify again.
Normally you would expect writing duplicate blocks dedup'd to be faster than
writing the non-dedup'd data, because you get to skip the actual write. (This
idealistic performance gain might be pure vapor due to need to update metadata,
but ignoring that technicality, continue hypothetically...) When you verify,
supposedly you're giving up that hypothetical performance gain, because you
have to do a read instead of a write. So at first blush, it seems like no
net-gain for performance. But if the duplicate block gets written frequently
(for example a block of all zeros) then there's a high probability the
duplicate block is already in ARC, so you can actually skip reading from disk
and just read from RAM.
So, the "10 extra reads" will sometimes be true - if the duplicate block
doesn't already exist in ARC. And the "10 extra reads" will sometimes be false
- if the duplicate block is already in ARC.
zfs-discuss mailing list