Re: [zfs-discuss] RAIDZ2 configuration
Hi Bruno, I agree that the raidz2 example on this page is weak and I will provide a better one. ZFS is very flexible and can be configured many different ways. If someone new to ZFS wants to take 3 old (but reliable) disks and make a raidz2 configuration for testing, we would not consider this is a nonsensical idea. If I had only 3 disks, I would create a mirrored configuration of two disks and keep one as a spare. Thanks, Cindy On 03/31/10 16:02, Bruno Sousa wrote: Hi Cindy, This all issue started when i asked opinion in this list in how should i create zpools. It seems that one of my initial ideas of creating a vdev with 3 disks in a raidz configuration seems to be a non-sense configuration. Somewhere along the way i defended my initial idea with the fact that the documentation from Sun has as an example such configuration as seen here : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* at http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view So if by concept the idea of having a vdev with 3 disks within a raidz configuration is a bad one, the oficial Sun documentation should not have such example. However if people made such example in Sun documentation, perhaps this all idea is not that bad at all.. Can you provide anything on this subject? Thanks, Bruno On 31-3-2010 23:49, Cindy Swearingen wrote: Hi Ned, If you look at the examples on the page that you cite, they start with single-parity RAIDZ examples and then move to double-parity RAIDZ example with supporting text, here: http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view Can you restate the problem with this page? Thanks, Cindy On 03/26/10 05:42, Edward Ned Harvey wrote: Just because most people are probably too lazy to click the link, I’ll paste a phrase from that sun.com webpage below: “Creating a single-parity RAID-Z pool is identical to creating a mirrored pool, except that the ‘raidz’ or ‘raidz1’ keyword is used instead of ‘mirror’.” And “zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0” So … Shame on you, Sun, for doing this to your poor unfortunate readers. It would be nice if the page were a wiki, or somehow able to have feedback submitted… *From:* zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] *On Behalf Of *Bruno Sousa *Sent:* Thursday, March 25, 2010 3:28 PM *To:* Freddie Cash *Cc:* ZFS filesystem discussion list *Subject:* Re: [zfs-discuss] RAIDZ2 configuration Hmm...it might be completely wrong , but the idea of raidz2 vdev with 3 disks came from the reading of http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view . This particular page has the following example : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* # *zpool status -v tank* pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c2t0d0ONLINE 0 0 0 c3t0d0ONLINE 0 0 0 So...what am i missing here? Just a bad example in the sun documentation regarding zfs? Bruno On 25-3-2010 20:10, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com mailto:bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com mailto:fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org mailto:zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Hi Bruno, I agree that the raidz2 example on this page is weak and I will provide a better one. ZFS is very flexible and can be configured many different ways. If someone new to ZFS wants to take 3 old (but reliable) disks and make a raidz2 configuration for testing, we would not consider this is a nonsensical idea. You can then apply what you learn about ZFS space allocation and redundancy to a new configuration. If I had only 3 disks, I would create a mirrored configuration of two disks and keep one as a spare. Thanks, Cindy On 03/31/10 16:02, Bruno Sousa wrote: Hi Cindy, This all issue started when i asked opinion in this list in how should i create zpools. It seems that one of my initial ideas of creating a vdev with 3 disks in a raidz configuration seems to be a non-sense configuration. Somewhere along the way i defended my initial idea with the fact that the documentation from Sun has as an example such configuration as seen here : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* at http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view So if by concept the idea of having a vdev with 3 disks within a raidz configuration is a bad one, the oficial Sun documentation should not have such example. However if people made such example in Sun documentation, perhaps this all idea is not that bad at all.. Can you provide anything on this subject? Thanks, Bruno On 31-3-2010 23:49, Cindy Swearingen wrote: Hi Ned, If you look at the examples on the page that you cite, they start with single-parity RAIDZ examples and then move to double-parity RAIDZ example with supporting text, here: http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view Can you restate the problem with this page? Thanks, Cindy On 03/26/10 05:42, Edward Ned Harvey wrote: Just because most people are probably too lazy to click the link, I’ll paste a phrase from that sun.com webpage below: “Creating a single-parity RAID-Z pool is identical to creating a mirrored pool, except that the ‘raidz’ or ‘raidz1’ keyword is used instead of ‘mirror’.” And “zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0” So … Shame on you, Sun, for doing this to your poor unfortunate readers. It would be nice if the page were a wiki, or somehow able to have feedback submitted… *From:* zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] *On Behalf Of *Bruno Sousa *Sent:* Thursday, March 25, 2010 3:28 PM *To:* Freddie Cash *Cc:* ZFS filesystem discussion list *Subject:* Re: [zfs-discuss] RAIDZ2 configuration Hmm...it might be completely wrong , but the idea of raidz2 vdev with 3 disks came from the reading of http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view . This particular page has the following example : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* # *zpool status -v tank* pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c2t0d0ONLINE 0 0 0 c3t0d0ONLINE 0 0 0 So...what am i missing here? Just a bad example in the sun documentation regarding zfs? Bruno On 25-3-2010 20:10, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com mailto:bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com mailto:fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org mailto:zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs
Re: [zfs-discuss] RAIDZ2 configuration
Cindy Swearingen wrote: If someone new to ZFS wants to take 3 old (but reliable) disks and make a raidz2 configuration for testing, we would not consider this is a nonsensical idea. You can then apply what you learn about ZFS space allocation and redundancy to a new configuration. Nonsensical may be a bit strong, but I can see no possible use case where a 3 disk raidz2 isn't better served by a 3-way mirror. Of course many things that are terrible ideas in production become useful learning experiences ;-) -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Thu, Apr 1, 2010 at 11:46 AM, Carson Gaspar car...@taltos.org wrote: Nonsensical may be a bit strong, but I can see no possible use case where a 3 disk raidz2 isn't better served by a 3-way mirror. Once bp_rewrite is done, you'll be able add disks to the raidz2. I suppose that's one reason? -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Brandon High wrote: On Thu, Apr 1, 2010 at 11:46 AM, Carson Gaspar car...@taltos.org mailto:car...@taltos.org wrote: Nonsensical may be a bit strong, but I can see no possible use case where a 3 disk raidz2 isn't better served by a 3-way mirror. Once bp_rewrite is done, you'll be able add disks to the raidz2. I suppose that's one reason? If you can expand a raidz2, you should be able to convert a mirror into a raidz2. Actually, you can do that now (with some trickery): - zpool detach oldpool exmirror3 - Add your new disk(s) to the system - mkfile /blah/fake1; mkfile /blah/fake2 - zpool create newpool raidz2 exmirror3 newdisk1 [newdisk2...] fake1 fake2 - zpool offline newpool fake1 fake2 - copy oldpool to newpool with zfs send/recv - zpool detach oldpool exmirror2 - zpool replace newpool fake2 exmirror2 - zpool destroy oldpool - zpool replace newpool fake1 exmirror1 3-way mirror becomes 4 or more disk raidz2 while retaining redundancy except for the brief period when you are running with a single disk old pool and a resilvering raidz2, and even then you have 2 copies of your data. Sneaky, eh? -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Hi Ned, If you look at the examples on the page that you cite, they start with single-parity RAIDZ examples and then move to double-parity RAIDZ example with supporting text, here: http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view Can you restate the problem with this page? Thanks, Cindy On 03/26/10 05:42, Edward Ned Harvey wrote: Just because most people are probably too lazy to click the link, I’ll paste a phrase from that sun.com webpage below: “Creating a single-parity RAID-Z pool is identical to creating a mirrored pool, except that the ‘raidz’ or ‘raidz1’ keyword is used instead of ‘mirror’.” And “zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0” So … Shame on you, Sun, for doing this to your poor unfortunate readers. It would be nice if the page were a wiki, or somehow able to have feedback submitted… *From:* zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] *On Behalf Of *Bruno Sousa *Sent:* Thursday, March 25, 2010 3:28 PM *To:* Freddie Cash *Cc:* ZFS filesystem discussion list *Subject:* Re: [zfs-discuss] RAIDZ2 configuration Hmm...it might be completely wrong , but the idea of raidz2 vdev with 3 disks came from the reading of http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view . This particular page has the following example : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* # *zpool status -v tank* pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c2t0d0ONLINE 0 0 0 c3t0d0ONLINE 0 0 0 So...what am i missing here? Just a bad example in the sun documentation regarding zfs? Bruno On 25-3-2010 20:10, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com mailto:bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com mailto:fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org mailto:zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Hi Cindy, This all issue started when i asked opinion in this list in how should i create zpools. It seems that one of my initial ideas of creating a vdev with 3 disks in a raidz configuration seems to be a non-sense configuration. Somewhere along the way i defended my initial idea with the fact that the documentation from Sun has as an example such configuration as seen here : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* at http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view So if by concept the idea of having a vdev with 3 disks within a raidz configuration is a bad one, the oficial Sun documentation should not have such example. However if people made such example in Sun documentation, perhaps this all idea is not that bad at all.. Can you provide anything on this subject? Thanks, Bruno On 31-3-2010 23:49, Cindy Swearingen wrote: Hi Ned, If you look at the examples on the page that you cite, they start with single-parity RAIDZ examples and then move to double-parity RAIDZ example with supporting text, here: http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view Can you restate the problem with this page? Thanks, Cindy On 03/26/10 05:42, Edward Ned Harvey wrote: Just because most people are probably too lazy to click the link, I’ll paste a phrase from that sun.com webpage below: “Creating a single-parity RAID-Z pool is identical to creating a mirrored pool, except that the ‘raidz’ or ‘raidz1’ keyword is used instead of ‘mirror’.” And “zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0” So … Shame on you, Sun, for doing this to your poor unfortunate readers. It would be nice if the page were a wiki, or somehow able to have feedback submitted… *From:* zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] *On Behalf Of *Bruno Sousa *Sent:* Thursday, March 25, 2010 3:28 PM *To:* Freddie Cash *Cc:* ZFS filesystem discussion list *Subject:* Re: [zfs-discuss] RAIDZ2 configuration Hmm...it might be completely wrong , but the idea of raidz2 vdev with 3 disks came from the reading of http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view . This particular page has the following example : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* # *zpool status -v tank* pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c2t0d0ONLINE 0 0 0 c3t0d0ONLINE 0 0 0 So...what am i missing here? Just a bad example in the sun documentation regarding zfs? Bruno On 25-3-2010 20:10, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com mailto:bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com mailto:fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org mailto:zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss smime.p7s Description: S/MIME Cryptographic Signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode. Freddie, are you nuts? This is false. Sure you can use raidz2 with 3 disks in it. But it does seem pointless to do that instead of a 3-way mirror. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Coolio. Learn something new everyday. One more way that raidz is different from RAID5/6/etc. Freddie, again, you're wrong. Yes, it's perfectly acceptable to create either raid-5 or raidz using 2 disks. It's not degraded, but it does seem pointless to do this instead of a mirror. Likewise, it's perfectly acceptable to create a raid-6 or raid-dp or raidz2 using 3 disks. It's not degraded, but seems pointless to do this instead of a 3-way mirror. Since it's pointless, some hardware vendors may not implement it in their raid controllers. They might only give you the option of creating a mirror instead. But that doesn't mean it's invalid raid configuration. So, is it just a standard that hardware/software RAID setups require 3 drives for a RAID5 array? And 4 drives for RAID6? It is just standard not to create a silly 2-disk raid5 or raidz. But don't use the word require. It is common practice to create raidz2 only with 4 disks or more, but again, don't use the word require. Some people do in fact create these silly configurations just because they're unfamiliar with what it all means. Take Bruno's original post as example, and that article he referenced on sun.com. How these things get started, I'll never know. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Just because most people are probably too lazy to click the link, I’ll paste a phrase from that sun.com webpage below: “Creating a single-parity RAID-Z pool is identical to creating a mirrored pool, except that the ‘raidz’ or ‘raidz1’ keyword is used instead of ‘mirror’.” And “zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0” So … Shame on you, Sun, for doing this to your poor unfortunate readers. It would be nice if the page were a wiki, or somehow able to have feedback submitted… From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Bruno Sousa Sent: Thursday, March 25, 2010 3:28 PM To: Freddie Cash Cc: ZFS filesystem discussion list Subject: Re: [zfs-discuss] RAIDZ2 configuration Hmm...it might be completely wrong , but the idea of raidz2 vdev with 3 disks came from the reading of http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view . This particular page has the following example : zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0 # zpool status -v tank pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c2t0d0ONLINE 0 0 0 c3t0d0ONLINE 0 0 0 So...what am i missing here? Just a bad example in the sun documentation regarding zfs? Bruno On 25-3-2010 20:10, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by http://www.mailscanner.info/ MailScanner, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Fri, Mar 26 at 7:29, Edward Ned Harvey wrote: Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode. Freddie, are you nuts? This is false. Sure you can use raidz2 with 3 disks in it. But it does seem pointless to do that instead of a 3-way mirror. One thing about mirrors is you can put each side of your mirror on a different controller, so that any single controller failure doesn't cause your pool to go down. While controller failure rates are very low, using 16/24 or 14/21 drives for parity on a dataset seems crazy to me. I know disks can be unreliable, but they shouldn't be THAT unreliable. I'd think that spending fewer drives for hot redundancy and then spending some of the balance on an isolated warm/cold backup solution would be more cost effective. http://blog.richardelling.com/2010/02/zfs-data-protection-comparison.html Quoting from the summary, at some point, the system design will be dominated by common failures and not the failure of independent disks. Another thought is that if heavy seeking is more likely to lead to high temperature and/or drive failure, then reserving one or two slots for an SSD L2ARC might be a good idea. It'll take a lot of load off of your spindles if your data set fits or mostly fits within the L2ARC. You'd need a lot of RAM to make use of a large L2ARC though, just something to keep in mind. We have a 32GB X25-E as L2ARC and though it's never more than ~5GB full with our workloads, most every file access saturates the wire (1.0 Gb/s ethernet) once the cache has warmed up, resulting in very little IO to our spindles. --eric -- Eric D. Mudama edmud...@mail.bounceswoosh.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Fri, March 26, 2010 07:38, Edward Ned Harvey wrote: Coolio. Learn something new everyday. One more way that raidz is different from RAID5/6/etc. Freddie, again, you're wrong. Yes, it's perfectly acceptable to create either raid-5 or raidz using 2 disks. It's not degraded, but it does seem pointless to do this instead of a mirror. I think the word you're looking for is possible, not acceptable. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] RAIDZ2 configuration
Hi all, Yet another question regarding raidz configuration.. Assuming a system with 24 disks available , having in mind reliability as the crucial factor , secondary the usable space and finally performance would be the last criteria, what would be the preferable configuration ? Should it be : * A - 7 raidz2 groups with 3 disks each and 3 disks as hot-spares * B - 3 raidz2 groups with 7 disks each and 3 disks as hot-spares * C - 5 raidz2 groups with 4 disks each and 4 disks as hot-spares * D - 4 raidz2 groups with 5 disks each and 4 disks as hot-spares * E - other? ;) Thanks for all your attention, Bruno smime.p7s Description: S/MIME Cryptographic Signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
I think I would do 3xraidz3 with 8 disks and 0 hotspares. That way you have a better chance of resolving bit rot issues that might become apparent during a rebuild. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Thu, Mar 25, 2010 at 6:28 AM, Bruno Sousa bso...@epinfante.com wrote: Assuming a system with 24 disks available , having in mind reliability as the crucial factor , secondary the usable space and finally performance would be the last criteria, what would be the preferable configuration ? Should it be : - A - 7 raidz2 groups with 3 disks each and 3 disks as hot-spares - B - 3 raidz2 groups with 7 disks each and 3 disks as hot-spares - C - 5 raidz2 groups with 4 disks each and 4 disks as hot-spares - D - 4 raidz2 groups with 5 disks each and 4 disks as hot-spares - E - other? ;) Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode. Why do you want so many hot-spares? Are you really expecting that many drives to die simultaneously? 1-2 would be plenty for a 24 drive system, if even that many. We have a couple 24-drive storage servers running. They are currently using 3x raidz2 vdevs of 8 drives each, spread across two 12-port controllers (8 on one controller, 8 on the other, 8 spread across the two). If I was to re-do these servers today, I would go with 4x raidz2 vdevs of 6 drives each and either put 2 vdevs on each controller, or switch to using 4 separate 8-port controllers with 1 vdev per controller. We don't use hot-spares, as we have a lot of monitoring to detect when a drive dies or the pool becomes degraded, and have a stack of spare drives standing by to use as replacements. -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Hi, What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? Indeed it may be too many spares...the discussion here it's between myself (the new guy that embraces new technologies) vs a EMC lover sysadmin... Currently the system is configured with 4 vdevs each with 5 drives, and i'm considering to use the 4 others ones to create something a raid10 (usefull for the backup software indexes...) . So far the system seems to behave quite nice...but than again we are just starting it. Thanks for the input, Bruno On 25-3-2010 16:46, Freddie Cash wrote: On Thu, Mar 25, 2010 at 6:28 AM, Bruno Sousa bso...@epinfante.com mailto:bso...@epinfante.com wrote: Assuming a system with 24 disks available , having in mind reliability as the crucial factor , secondary the usable space and finally performance would be the last criteria, what would be the preferable configuration ? Should it be : * A - 7 raidz2 groups with 3 disks each and 3 disks as hot-spares * B - 3 raidz2 groups with 7 disks each and 3 disks as hot-spares * C - 5 raidz2 groups with 4 disks each and 4 disks as hot-spares * D - 4 raidz2 groups with 5 disks each and 4 disks as hot-spares * E - other? ;) Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode. Why do you want so many hot-spares? Are you really expecting that many drives to die simultaneously? 1-2 would be plenty for a 24 drive system, if even that many. We have a couple 24-drive storage servers running. They are currently using 3x raidz2 vdevs of 8 drives each, spread across two 12-port controllers (8 on one controller, 8 on the other, 8 spread across the two). If I was to re-do these servers today, I would go with 4x raidz2 vdevs of 6 drives each and either put 2 vdevs on each controller, or switch to using 4 separate 8-port controllers with 1 vdev per controller. We don't use hot-spares, as we have a lot of monitoring to detect when a drive dies or the pool becomes degraded, and have a stack of spare drives standing by to use as replacements. -- Freddie Cash fjwc...@gmail.com mailto:fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss smime.p7s Description: S/MIME Cryptographic Signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On 25-3-2010 15:28, Richard Jahnel wrote: I think I would do 3xraidz3 with 8 disks and 0 hotspares. That way you have a better chance of resolving bit rot issues that might become apparent during a rebuild. Indeed raidz3...i didn't consider it. In short, a raidz3 could sustain 3 broken drives per vdev vs 2 broken drives in a raidz2 pool ? Perfomance and usable space , how does it compares? Bruno smime.p7s Description: S/MIME Cryptographic Signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Bruno Sousa wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? Technically a 3 disk raidz2 won't be degraded, so I'm not sure what he meant. However a 3-way mirror will be much faster and equally reliable, so a 3 disk raidz2 doesn't make much sense. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Well the thing I like about raidz3 is that even with 1 drive out you have 3 copies of all the blocks. So if you encounter bit rot, not only can checksums be used to find the good data, you can still get a best 2 out of 3 vote on which data is correct. As to performance, all I can say is test test test. Pick your top 3 contenders, install bonnie++ and then test each configuration. Then make your decision based on your personal balance between performance, space and reliability. Due to space considerations I had to choose raidz2 over raidz3. I just couldn't give up that last drives worth of space. oddly enough in my enviroment I got better performance out of raidz2 than I did out of 7 mirrors striped together. It may be because they are all 250gb ssds. I think the bottleneck in my case is either the Adaptec raid card or cpu. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Hmm...it might be completely wrong , but the idea of raidz2 vdev with 3 disks came from the reading of http://docs.sun.com/app/docs/doc/819-5461/gcvjg?a=view . This particular page has the following example : *zpool create tank raidz2 c1t0d0 c2t0d0 c3t0d0* # *zpool status -v tank* pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t0d0ONLINE 0 0 0 c2t0d0ONLINE 0 0 0 c3t0d0ONLINE 0 0 0 So...what am i missing here? Just a bad example in the sun documentation regarding zfs? Bruno On 25-3-2010 20:10, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com mailto:bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). -- Freddie Cash fjwc...@gmail.com mailto:fjwc...@gmail.com -- This message has been scanned for viruses and dangerous content by *MailScanner* http://www.mailscanner.info/, and is believed to be clean. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss smime.p7s Description: S/MIME Cryptographic Signature ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Mar 25, 2010, at 22:10, Freddie Cash fjwc...@gmail.com wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) For any kind of RAID-Z as an absolute minimum you need one drive for data and 1, 2 or 3 drives for parity. IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive) No, it will not be degraded. You can easily check it with mkfile -n 64m /var/tmp/1 /var/tmp/2 /var/tmp/3 zpool create rz2 raidz2 /var/tmp/[123] zpool status rz2 Regards Victor -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Mar 25, 2010, at 12:10 PM, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) no. raidz requires a minimum of 2 drives: data + parity raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) Similarly, raidz2 requires 3 drives: data + 2 parity IOW, a raidz2 vdev made up of 3 drives will always be running in degraded mode (it's missing a drive). The definition of the degraded state is in the zpool(1m) man page: DEGRADED One or more top-level vdevs is in the degraded state because one or more component devices are offline. Suf- ficient replicas exist to continue functioning. One or more component devices is in the degraded or faulted state, but sufficient replicas exist to continue functioning. The underlying conditions are as follows: oThe number of checksum errors exceeds accept- able levels and the device is degraded as an indication that something may be wrong. ZFS continues to use the device as necessary. oThe number of I/O errors exceeds acceptable levels. The device could not be marked as faulted because there are insufficient replicas to continue functioning. -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
On Thu, Mar 25, 2010 at 12:44 PM, Richard Elling richard.ell...@gmail.comwrote: On Mar 25, 2010, at 12:10 PM, Freddie Cash wrote: On Thu, Mar 25, 2010 at 11:47 AM, Bruno Sousa bso...@epinfante.com wrote: What do you mean by Using fewer than 4 disks in a raidz2 defeats the purpose of raidz2, as you will always be in a degraded mode ? Does it means that having 2 vdevs with 3 disks it won't be redundant in the advent of a drive failure? raidz1 is similar to raid5 in that it is single-parity, and requires a minimum of 3 drives (2 data + 1 parity) no. raidz requires a minimum of 2 drives: data + parity raidz2 is similar to raid6 in that it is double-parity, and requires a minimum of 4 drives (2 data + 2 parity) Similarly, raidz2 requires 3 drives: data + 2 parity Coolio. Learn something new everyday. One more way that raidz is different from RAID5/6/etc. So, is it just a standard that hardware/software RAID setups require 3 drives for a RAID5 array? And 4 drives for RAID6? -- Freddie Cash fjwc...@gmail.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] RAIDZ2 configuration
Freddie Cash wrote: So, is it just a standard that hardware/software RAID setups require 3 drives for a RAID5 array? And 4 drives for RAID6? It's padding on the sharp edges. See my earlier post - a 2 disk RAID5 is silly, use a mirror. A 3 disk RAID6 is silly, use a 3-way mirror. Both are legal, but most vendors don't let you do it b/c you really don't want to. -- Carson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
