> On Tue, Sep 7, 2010 at 4:59 PM, Edward Ned Harvey <sh...@nedharvey.com> > wrote: > > I think the value you can take from this is: > Why does the BPG say that? What is the reasoning behind it? > > Anything that is a "rule of thumb" either has reasoning behind it (you > should know the reasoning) or it doesn't (you should ignore the rule of > thumb, dismiss it as myth.)
Let's examine the myth that you should limit the number of drives in a vdev because of resilver time. The myth goes something like this: You shouldn't use more than ___ drives in a vdev raidz_ configuration, because all the drives need to read during a resilver, so the more drives are present, the longer the resilver time. The truth of the matter is: Only the size of used data is read. Because this is ZFS, it's smarter than a hardware solution which would have to read all disks in their entirety. In ZFS, if you have a 6-disk raidz1 with capacity of 5 disks, and a total of 50G of data, then each disk has roughly 10G of data in it. During resilver, 5 disks will each read 10G of data, and 10G of data will be written to the new disk. If you have a 11-disk raidz1 with capacity of 10 disks, then each disk has roughly 5G of data. 10 disks will each read 5G of data, and 5G of data will be written to the new disk. If anything, more disks means a faster resilver, because you're more easily able to saturate the bus, and you have a smaller amount of data that needs to be written to the replaced disk. Let's examine the myth that you should limit the number of disks for the sake of redundancy. It is true that a carefully crafted system can survive things like SCSI controller or tray failure. Suppose you have 3 scsi cards. Suppose you construct a raidz2 device using 2 disks from controller 0, 2 disks from controller 1, and 2 disks from controller 2. Then if a controller dies, you have only lost 2 disks, and you are degraded but still functional as long as you don't lose another disk. But you said you have 20 disks all connected to a single controller. So none of that matters in your case. Personally, I can't imagine any good reason to generalize "don't use more than ___ devices in a vdev." To me, a 12-disk raidz2 is just as likely to fail as a 6-disk raidz1. But a 12-disk raidz2 is slightly more reliable than having two 6-disk raidz1's. Perhaps, maybe, a 64bit processor is able to calculate parity on an 8-disk raidz set in a single operation, but requires additional operations to calculate parity if your raidz has 9 or more disks in it ... But I am highly skeptical of this line of reasoning, and AFAIK, nobody has ever suggested this before me. I made it up just now. I'm grasping at straws and stretching my imagination to find *any* merit in the statement, "don't use more than ___ disks in a vdev." I see no reasoning behind it, and unless somebody can say anything to support it, I think it's bunk. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
