On 6/12/2011 5:08 AM, Dimitar Hadjiev wrote:
I can lay them out as 4*3-disk raidz1, 3*4-disk-raidz1
or a 1*12-disk raidz3 with nearly the same capacity (8-9
data disks plus parity). I see that with more vdevs the
IOPS will grow - does this translate to better resilver
and scrub times as well?
Yes it would translate in better resilver times as any failures will affect
only one of the vdevs leading to a shorter parity restore time as oposed to
rebuilding the whole raidz2. As for scrubbing it would be as fast as the scrub
of each vdev since the whole pool does not have parity data to synchronize.
Go look through the mail archives, and there's at least a couple of
posts from me and Richard Elling (amongst others) about the workload
that a resilver requires on a raidz* vdev. Essentially, "typical" usage
of a vdev will result in resilver times linearly degrading with each
additional DATA disk in the raidz*, as a resilver is IOPS-bound on the
single replaced disk. So, a 3-disk raidz1 (2 data disks) should, on
average, resilver 4.5 times faster than a 12-disk raidz3 (9 data disks).
How good or bad is the expected reliability of
3*4-disk-raidz1 vs 1*12-disk raidz3, so which
of the tradeoffs is better - more vdevs or more
parity to survive loss of ANY 3 disks vs. "right"
3 disks?
I'd say the chances of loseing a whole vdev in a 4*3 configuration equal the
chances of loseing 4 drives in a 1*12 raidz3 configuration - it might happen,
nothing is foolproof.
No, the reliability of a 1x12raidz3 is *significantly* better than that
of 4x3 raidz1 (or, frankly, ANY raidz1 configuration using 12 disks).
Richard has some stats around here somewhere... basically, the math
(singular, you damn Brits! :-) says that while a 3-disk raidz1 will
certainly take shorter to re-silver after a loss than a 12-disk raidz3,
this is more than counterbalanced by the ability of a 12-disk raidz3 to
handle additional disk losses, where the 4x3 config is only
*probabilisticly* likely to be handle a 2nd or 3rd drive failure.
I'd have to re-look at the exact numbers, but, I'd generally say that
2x6raidz2 vdevs would be better than either 1x12raidz3 or 4x3raidz1 (or
3x4raidz1, for a home server not looking for super-critical protection
(in which case, you should be using mirrors with spares, not raidz*).
--
Erik Trimble
Java System Support
Mailstop: usca22-123
Phone: x17195
Santa Clara, CA
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
