On Jan 5, 2013, at 9:42 AM, Russ Poyner <rpoy...@engr.wisc.edu> wrote:

> I'm configuring a box with 24x 3Tb consumer SATA drives, and wondering about 
> the best way to configure the pool. The customer wants capacity on the cheap, 
> and I want something I can service without sweating too much about data loss. 
> Due to capacity concerns raid 10 is probably out, which leaves various raidz 
> choices

You should consider space, data dependability as measured by Mean Time to Data 
Loss (MTTDL), and performance.

For MTTDL[1] model, let's use 700k hours MTBF for the disks and 168 hours for 
recovery (48 hours logistical + 120 hours resilver of a full disk)

For performance, lets hope for 7,200 rpms and about 80 IOPS for small, random 
with 100% cache miss.

> 1. A stripe of four 6 disk raidz2

Option 1
        space ~= 4 * (6 - 2) * 3TB = 48 TB
        MTTDL[1] = 8.38e+5 years or  0.000119% Annualized Failure Rate (AFR)
        small, random read performance, best of worst case = 4 * (6/4) * 80 

> 2. A stripe of two 11 disk raidz3 with 2 hot spares.

Option 2
        space ~= 2 * (11 - 3) * 3TB = 48 TB
        MTTDL[1] =  3.62e+7 years or 0.000003% AFR 
        small, random read performance, best of worst case = 2 * (11/8) * 80 
IOPS =  220 IOPS

Option 2a (no hot spares)
        space ~= 2 * (12 - 3) * 3TB = 54 TB
        MTTDL[2] = 1.90e+7 years or 0.000005% AFR
        small, random read performance, best of worst case = 2 * (12/9) * 80 
IOPS =  213 IOPS

> Other, better ideas?

There are thousands of permutations you could consider :-)
For 24-bay systems with double parity or better, we also see a 3x8-disk as a
common configuration. Offhand, I'd say we see more 4x6-disk and 3x8-disk
configs than any configs with more than 10 disks per set.

> My questions are
> A. How long will resilvering take with these layouts when the disks start 
> dying?

It depends on the concurrent workload. By default resilvers are throttled and 
way to other workload. In general, for double or triple parity RAID, you don't 
to worry too much on a per-disk basis. The conditions you need to worry about 
where the failure cause is common to all disks, such as a controller, fans, 
or power because they are more likely than a triple failure of disks (as 
clearly shown
by the MTTDL[1] model results above)

> B. Should I prefer hot spares or additional parity drives, and why?

In general, addional parity is better than hot spares. You get more performance
and better data dependability.

> The box is a supermicro with 36 bays controlled through a single LSI 9211-8i. 
> There is a separate intel 320 ssd for the OS. The purpose is to backup data 
> from the customer's windows workstations. I'm leaning toward using BackupPC 
> for the backups since it seems to combine good efficiency with a fairly 
> customer-friendly web interface.

Sounds like a good plan.
 -- richard

> I'm running FreeBSD 9, after having failed to get the plugin jail working in 
> FreeNAS, also for personal reasons I find csh easier to use than the FreeNAS 
> web interface. My impression is that FreeBSD combines a mature OS with the 
> second oldest/best (after Illumos) free implementation of zfs.
> Thanks in advance
> Russ Poyner
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss



zfs-discuss mailing list

Reply via email to