Re: [Gluster-users] To RAID or not to RAID...
Hi, our old setup is not really comparable, but i thought i'd drop some lines... we once had a Distributed-Replicate setup with 4 x 3 = 12 disks (10 TB hdd). Simple JBOD, every disk == brick. Was running pretty good, until one of the disks died. The restore (reset-brick) took about a month, because the application has a quite high I/O and therefore slows down the volume and the disks. Next step: take servers with 10x10TB disks and build a RAID 10; raid array == brick, replicate volume (1 x 3 = 3). When a disk fails, you only have to rebuild the SW RAID which takes about 3-4 days, plus the periodic redundany checks. This was way better than the JBOD/reset-scenario before. But still not optimal. Upcoming step: build a distribute-replicate with lots of SSDs (maybe again with a RAID underneath) . tl;dr what i wanted to say: we waste a lot of disks. It simply depends on which setup you have and how to handle the situation when one of the disks fails - and it will! ;-( regards Hubert Am Di., 14. Jan. 2020 um 12:36 Uhr schrieb Markus Kern : > > > Greetings again! > > After reading RedHat documentation regarding optimizing Gluster storage > another question comes to my mind: > > Let's presume that I want to go the distributed dispersed volume way. > Three nodes which two bricks each. > According to RedHat's recommendation, I should use RAID6 as underlying > RAID for my planned workload. > I am frightened by that "waste" of disks in such a case: > When each brick is a RAID6, I would "loose" two disks per brick - 12 > lossed disks in total. > In addition to this, distributed dispersed volume adds another layer of > lossed disk space. > > Am I wrong here? Maybe I didn't understand the recommendations wrong? > > Markus > > > Community Meeting Calendar: > > APAC Schedule - > Every 2nd and 4th Tuesday at 11:30 AM IST > Bridge: https://bluejeans.com/441850968 > > NA/EMEA Schedule - > Every 1st and 3rd Tuesday at 01:00 PM EDT > Bridge: https://bluejeans.com/441850968 > > Gluster-users mailing list > Gluster-users@gluster.org > https://lists.gluster.org/mailman/listinfo/gluster-users Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] To RAID or not to RAID...
Hi Markus, You are right .I think that the 3 node setup matches distributed volume. According to https://docs.gluster.org/en/latest/Administrator%20Guide/Setting%20Up%20Volumes/ Dispersed volumes use erasure codes to have a 'parity' on a separate brick. In such case you can afford to loose a brick without loosing data and you will need more bricks. Yet, I don't see anything about RAID6 being required. Use the gluster's official documentation (if possible) as it is the most recent info. Maybe you can share the ammount of disks , raid controllers and servers you have and your tolerance to data loss. Then I can share my thoughts on the possible volume types. Best Regards, Strahil Nikolov On Jan 14, 2020 20:33, Markus Kern wrote: > > Hi Strahil, > > thanks for you answer - but now I am completely lost :) > > From this documentation: > https://docs.oracle.com/cd/E52668_01/F10040/html/gluster-312-volume-distr-disp.html > > > "As a dispersed volume must have a minimum of three bricks, a > distributed dispersed volume must have at least six bricks. For example, > six nodes with one brick, or three nodes with two bricks on each node > are needed for this volume type." > > So for a distributed dispersed volume I need at least six bricks. If > each brick is a RAID6, I have 6 x 2 Parity disks = 12 disks for parity. > > In your example you only have one brick per node in a three node setup. > This is no distributed dispersed volume then, right? > > A confused Markus > > > Am 14.01.2020 16:29, schrieb Strahil: > > Hi Markus, > > > > Distributed dispersed volume is just LVM's linear LV -> so in case of > > brick failiure - you loose the data on it. > > > > Raid 6 requires 2 disks for parity, so you can make a large RAID6 > > and use that as a single brick - so the disks that hold the parity > > data are only 6 ( 3 nodes x 2 disks). > > > > Of course if you have too many disks for a single raid controller > > ,that you can consider a replica volume with an arbiter. > > > > > > Best Regards, > > Strahil Nikolov > > > > On Jan 14, 2020 13:36, Markus Kern wrote: > >> > >> > >> Greetings again! > >> > >> After reading RedHat documentation regarding optimizing Gluster > >> storage > >> another question comes to my mind: > >> > >> Let's presume that I want to go the distributed dispersed volume way. > >> Three nodes which two bricks each. > >> According to RedHat's recommendation, I should use RAID6 as underlying > >> RAID for my planned workload. > >> I am frightened by that "waste" of disks in such a case: > >> When each brick is a RAID6, I would "loose" two disks per brick - 12 > >> lossed disks in total. > >> In addition to this, distributed dispersed volume adds another layer > >> of > >> lossed disk space. > >> > >> Am I wrong here? Maybe I didn't understand the recommendations wrong? > >> > >> Markus > >> > >> > >> Community Meeting Calendar: > >> > >> APAC Schedule - > >> Every 2nd and 4th Tuesday at 11:30 AM IST > >> Bridge: https://bluejeans.com/441850968 > >> > >> NA/EMEA Schedule - > >> Every 1st and 3rd Tuesday at 01:00 PM EDT > >> Bridge: https://bluejeans.com/441850968 > >> > >> Gluster-users mailing list > >> Gluster-users@gluster.org > >> https://lists.gluster.org/mailman/listinfo/gluster-users Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
Re: [Gluster-users] To RAID or not to RAID...
Hi Strahil, thanks for you answer - but now I am completely lost :) From this documentation: https://docs.oracle.com/cd/E52668_01/F10040/html/gluster-312-volume-distr-disp.html "As a dispersed volume must have a minimum of three bricks, a distributed dispersed volume must have at least six bricks. For example, six nodes with one brick, or three nodes with two bricks on each node are needed for this volume type." So for a distributed dispersed volume I need at least six bricks. If each brick is a RAID6, I have 6 x 2 Parity disks = 12 disks for parity. In your example you only have one brick per node in a three node setup. This is no distributed dispersed volume then, right? A confused Markus Am 14.01.2020 16:29, schrieb Strahil: Hi Markus, Distributed dispersed volume is just LVM's linear LV -> so in case of brick failiure - you loose the data on it. Raid 6 requires 2 disks for parity, so you can make a large RAID6 and use that as a single brick - so the disks that hold the parity data are only 6 ( 3 nodes x 2 disks). Of course if you have too many disks for a single raid controller ,that you can consider a replica volume with an arbiter. Best Regards, Strahil Nikolov On Jan 14, 2020 13:36, Markus Kern wrote: Greetings again! After reading RedHat documentation regarding optimizing Gluster storage another question comes to my mind: Let's presume that I want to go the distributed dispersed volume way. Three nodes which two bricks each. According to RedHat's recommendation, I should use RAID6 as underlying RAID for my planned workload. I am frightened by that "waste" of disks in such a case: When each brick is a RAID6, I would "loose" two disks per brick - 12 lossed disks in total. In addition to this, distributed dispersed volume adds another layer of lossed disk space. Am I wrong here? Maybe I didn't understand the recommendations wrong? Markus Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users Community Meeting Calendar: APAC Schedule - Every 2nd and 4th Tuesday at 11:30 AM IST Bridge: https://bluejeans.com/441850968 NA/EMEA Schedule - Every 1st and 3rd Tuesday at 01:00 PM EDT Bridge: https://bluejeans.com/441850968 Gluster-users mailing list Gluster-users@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users
