Re: RAID5 to RAID6 reshape?
On Sat, 16 Feb 2008 20:58:07 -0700, Beolach [EMAIL PROTECTED] said: beolach [ ... ] start w/ 3 drives in RAID5, and add drives as I beolach run low on free space, eventually to a total of 14 beolach drives (the max the case can fit). Like for for so many other posts to this list, all that is syntactically valid is not necessarily the same thing as that which is wise. beolach But when I add the 5th or 6th drive, I'd like to switch beolach from RAID5 to RAID6 for the extra redundancy. Again, what may be possible is not necessarily what may be wise. In particular it seems difficult to discern which usage such arrays would be put to. There might be a bit of difference between a giant FAT32 volume containing song lyrics files or an XFS filesystem with a collection of 500GB tomography scans in them cached from a large tape backup system. beolach I'm also interested in hearing people's opinions about beolach LVM / EVMS. They are yellow, and taste of vanilla :-). To say something more specific is difficult without knowing what kind of requirement they may be expected to satisfy. beolach I'm currently planning on just using RAID w/out the beolach higher level volume management, as from my reading I beolach don't think they're worth the performance penalty, [ beolach ... ] Very amusing that someone who is planning to grow a 3 drive RAID5 into a 14 drive RAID6 worries about the DM performance penalty. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
Beolach said: (by the date of Sat, 16 Feb 2008 20:58:07 -0700) I'm also interested in hearing people's opinions about LVM / EVMS. With LVM it will be possible for you to have several raid5 and raid6: eg: 5 HHDs (raid6), 5HDDs (raid6) and 4 HDDs (raid5). Here you would have 14 HDDs and five of them being extra - for safety/redundancy purposes. LVM allows you to join several blockdevices and create one huge partition on top of them. Without LVM you will end up with raid6 on 14 HDDs thus having only 2 drives used for redundancy. Quite risky IMHO. It is quite often that a *whole* IO controller dies and takes all 4 drives with it. So when you connect your drives, always make sure that you are totally safe if any of your IO conrollers dies (taking down 4 HDDs with it). With 5 redundant discs this may be possible to solve. Of course when you replace the controller the discs are up again, and only need to resync (which is done automatically). LVM can be grown on-line (without rebooting the computer) to join new block devices. And after that you only `resize2fs /dev/...` and your partition is bigger. Also in such configuration I suggest you to use ext3 fs, because no other fs (XFS, JFS, whatever) had that much testing than ext* filesystems had. Question to other people here - what is the maximum partition size that ext3 can handle, am I correct it 4 TB ? And to go above 4 TB we need to use ext4dev, right? best regards -- Janek Kozicki | - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Create Raid6 with 1 missing member fails
I tried to create a raid6 with one missing member, but it fails. It works fine to create a raid6 with two missing members. Is it supposed to be like that ? mdadm -C /dev/md0 -n5 -l6 -c256 /dev/sd[bcde]1 missing raid5: failed to run raid set md0 mdadm: RUN_ARRAY failed: Input/output error mdadm: stopped /dev/md0 mdadm --version mdadm - v2.6.4 - 19th October 2007 uname -a Linux compaq2.lan 2.6.23.15-137.fc8 #1 SMP Sun Feb 10 17:03:13 EST 2008 x86_64 x86_64 x86_64 GNU/Linux mdadm --examine /dev/sdb1 /dev/sdb1: Magic : a92b4efc Version : 00.90.00 UUID : e4147673:e22e02ac:09e8c875:b3e364c1 Creation Time : Sun Feb 17 14:31:55 2008 Raid Level : raid6 Used Dev Size : 488295424 (465.67 GiB 500.01 GB) Array Size : 1464886272 (1397.02 GiB 1500.04 GB) Raid Devices : 5 Total Devices : 5 Preferred Minor : 0 Update Time : Sun Feb 17 14:31:55 2008 State : active Active Devices : 4 Working Devices : 4 Failed Devices : 1 Spare Devices : 0 Checksum : d9c93170 - correct Events : 0.1 Chunk Size : 256K Number Major Minor RaidDevice State this 0 8 170 active sync /dev/sdb1 0 0 8 170 active sync /dev/sdb1 1 1 8 331 active sync /dev/sdc1 2 2 8 492 active sync /dev/sdd1 3 3 8 653 active sync /dev/sde1 4 4 004 faulty - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
Beolach said: (by the date of Sat, 16 Feb 2008 20:58:07 -0700) Or would I be better off starting w/ 4 drives in RAID6? oh, right - Sevrin Robstad has a good idea to solve your problem - create raid6 with one missing member. And add this member, when you have it, next year or such. -- Janek Kozicki | - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
suns raid-z / zfs
Hi any opinions on suns zfs/raid-z? It seems like a good way to avoid the performance problems of raid-5 /raid-6 But does it stripe? One could think that rewriting stripes other places would damage the striping effects. Or is the performance only meant to be good for random read/write? Can the code be lifted to Linux? I understand that it is already in freebsd. Does Suns licence prevent this? And could something like this be built into existing file systems like ext3 and xfs? They could have a multipartition layer in their code, and then the heuristics to optimize block access could also apply to stripe access. best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
On Sun, 17 Feb 2008 14:31:22 +0100 Janek Kozicki [EMAIL PROTECTED] wrote: Beolach said: (by the date of Sat, 16 Feb 2008 20:58:07 -0700) I'm also interested in hearing people's opinions about LVM / EVMS. With LVM it will be possible for you to have several raid5 and raid6: eg: 5 HHDs (raid6), 5HDDs (raid6) and 4 HDDs (raid5). Here you would have 14 HDDs and five of them being extra - for safety/redundancy purposes. LVM allows you to join several blockdevices and create one huge partition on top of them. Without LVM you will end up with raid6 on 14 HDDs thus having only 2 drives used for redundancy. Quite risky IMHO. I guess I'm just too reckless a guy. I don't like having wasted space, even though I know redundancy is by no means a waste. And part of me keeps thinking that the vast majority of my drives have never failed (although a few have, including one just recently, which is a large part of my motivation for this fileserver). So I was thinking RAID6, possibly w/ a hot spare or 2, would be safe enough. Speaking of hot spares, how well would cheap external USB drives work as hot spares? Is that a pretty silly idea? It is quite often that a *whole* IO controller dies and takes all 4 drives with it. So when you connect your drives, always make sure that you are totally safe if any of your IO conrollers dies (taking down 4 HDDs with it). With 5 redundant discs this may be possible to solve. Of course when you replace the controller the discs are up again, and only need to resync (which is done automatically). That sounds scary. Does a controller failure often cause data loss on the disks? My understanding was that one of the advantages of Linux's SW RAID was that if a controller failed you could swap in another controller, not even the same model or brand, and Linux would reassemble the RAID. But if a controller failure typically takes all the data w/ it, then the portability isn't as awesome an advantage. Is your last sentence about replacing the controller applicable to most controller failures, or just w/ more redundant discs? In my situation downtime is only mildly annoying, data loss would be much worse. LVM can be grown on-line (without rebooting the computer) to join new block devices. And after that you only `resize2fs /dev/...` and your partition is bigger. Also in such configuration I suggest you to use ext3 fs, because no other fs (XFS, JFS, whatever) had that much testing than ext* filesystems had. Plain RAID5 RAID6 are also capable of growing on-line, although I expect it's a much more complex time-consuming process than LVM. I had been planning on using XFS, but I could rethink that. Have there been many horror stories about XFS? Question to other people here - what is the maximum partition size that ext3 can handle, am I correct it 4 TB ? And to go above 4 TB we need to use ext4dev, right? I thought it depended on CPU architecture kernel version, w/ recent kernels on 64-bit archs being capable of 32 TiB. If it is only 4 TiB, I would go w/ XFS. oh, right - Sevrin Robstad has a good idea to solve your problem - create raid6 with one missing member. And add this member, when you have it, next year or such. I thought I read that would involve a huge performance hit, since then everything would require parity calculations. Or would that just be w/ 2 missing drives? Thanks, Conway S. Smith - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
I'm also interested in hearing people's opinions about LVM / EVMS. With LVM it will be possible for you to have several raid5 and raid6: eg: 5 HHDs (raid6), 5HDDs (raid6) and 4 HDDs (raid5). Here you would have 14 HDDs and five of them being extra - for safety/redundancy purposes. that's a very high price to pay. partition on top of them. Without LVM you will end up with raid6 on 14 HDDs thus having only 2 drives used for redundancy. Quite risky IMHO. your risk model is quite strange - 5/14 redundancy means that either you expect a LOT of failures, or you put a huge premium on availability. the latter is odd because normally, HA people go for replication of more components, not just controllers (ie, whole servers). It is quite often that a *whole* IO controller dies and takes all 4 you appear to be using very flakey IO controllers. are you specifically talking about very cheap ones, or in hostile environments? drives with it. So when you connect your drives, always make sure that you are totally safe if any of your IO conrollers dies (taking IO controllers are not a common failure mode, in my experience. when it happens, it usually indicates an environmental problem (heat, bad power, bad hotplug, etc). Question to other people here - what is the maximum partition size that ext3 can handle, am I correct it 4 TB ? 8 TB. people who want to push this are probably using ext4 already. And to go above 4 TB we need to use ext4dev, right? or patches (which have been around and even in some production use for a long while.) - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
Mark Hahn said: (by the date of Sun, 17 Feb 2008 17:40:12 -0500 (EST)) I'm also interested in hearing people's opinions about LVM / EVMS. With LVM it will be possible for you to have several raid5 and raid6: eg: 5 HHDs (raid6), 5HDDs (raid6) and 4 HDDs (raid5). Here you would have 14 HDDs and five of them being extra - for safety/redundancy purposes. that's a very high price to pay. partition on top of them. Without LVM you will end up with raid6 on 14 HDDs thus having only 2 drives used for redundancy. Quite risky IMHO. your risk model is quite strange - 5/14 redundancy means that either yeah, sorry. I went too far. I didn't have IO controller failure so far. But I've read about one on this list, and that all data was lost. You're right, better to duplicate a server with backup copy, so it is independent of the original one. -- Janek Kozicki | - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
On Sunday February 17, [EMAIL PROTECTED] wrote: On Sun, 17 Feb 2008 14:31:22 +0100 Janek Kozicki [EMAIL PROTECTED] wrote: oh, right - Sevrin Robstad has a good idea to solve your problem - create raid6 with one missing member. And add this member, when you have it, next year or such. I thought I read that would involve a huge performance hit, since then everything would require parity calculations. Or would that just be w/ 2 missing drives? A raid6 with one missing drive would have a little bit of a performance hit over raid5. Partly there is a CPU hit to calculate the Q block which is slower than calculating normal parity. Partly there is the fact that raid6 never does read-modify-write cycles, so to update one block in a stripe, it has to read all the other data blocks. But the worst aspect of doing this that if you have a system crash, you could get hidden data corruption. After a system crash you cannot trust parity data (as it may have been in the process of being updated) so you have to regenerate it from known good data. But if your array is degraded, you don't have all the known good data, so you loose. It is really best to avoid degraded raid4/5/6 arrays when at all possible. NeilBrown - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
On Saturday February 16, [EMAIL PROTECTED] wrote: found was a few months old. Is it likely that RAID5 to RAID6 reshaping will be implemented in the next 12 to 18 months (my rough Certainly possible. I won't say it is likely until it is actually done. And by then it will be definite :-) i.e. no concrete plans. It is always best to base your decisions on what is available today. NeilBrown - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: suns raid-z / zfs
On Sunday February 17, [EMAIL PROTECTED] wrote: Hi any opinions on suns zfs/raid-z? It's vaguely interesting. I'm not sold on the idea though. It seems like a good way to avoid the performance problems of raid-5 /raid-6 I think there are better ways. But does it stripe? One could think that rewriting stripes other places would damage the striping effects. I'm not sure what you mean exactly. But I suspect your concerns here are unjustified. Or is the performance only meant to be good for random read/write? I suspect it is mean to be good for everything. But you would have to ask SUN that. Can the code be lifted to Linux? I understand that it is already in freebsd. Does Suns licence prevent this? My understanding is that the sun license prevents it. However raid-z only makes sense in the context of a specific filesystem such as ZFS. It isn't something that you could just layer any filesystem on top of. And could something like this be built into existing file systems like ext3 and xfs? They could have a multipartition layer in their code, and then the heuristics to optimize block access could also apply to stripe access. I doubt it, but I haven't thought deeply enough about it to see if there might be some relatively non-intrusive way. NeilBrown best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Create Raid6 with 1 missing member fails
On Sunday February 17, [EMAIL PROTECTED] wrote: I tried to create a raid6 with one missing member, but it fails. It works fine to create a raid6 with two missing members. Is it supposed to be like that ? No, it isn't supposed to be like that, but currently it is. The easiest approach if to create it with 2 drives missing, and the extra drive immediately. This is essentially what mdadm will do when I fix it. Alternately you can use --assume-clean to tell it that the array is clean. It is actually a lie, but it is a harmless lie. Whenever any data is written to the array, that little part of the array will get cleaned. (Note that this isn't true of raid5, only of raid6). NeilBrown - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
On Sun, 17 Feb 2008 11:50:25 + [EMAIL PROTECTED] (Peter Grandi) wrote: On Sat, 16 Feb 2008 20:58:07 -0700, Beolach [EMAIL PROTECTED] said: beolach [ ... ] start w/ 3 drives in RAID5, and add drives as I beolach run low on free space, eventually to a total of 14 beolach drives (the max the case can fit). Like for for so many other posts to this list, all that is syntactically valid is not necessarily the same thing as that which is wise. Which part isn't wise? Starting w/ a few drives w/ the intention of growing; or ending w/ a large array (IOW, are 14 drives more than I should put in 1 array expect to be safe from data loss)? beolach But when I add the 5th or 6th drive, I'd like to switch beolach from RAID5 to RAID6 for the extra redundancy. Again, what may be possible is not necessarily what may be wise. In particular it seems difficult to discern which usage such arrays would be put to. There might be a bit of difference between a giant FAT32 volume containing song lyrics files or an XFS filesystem with a collection of 500GB tomography scans in them cached from a large tape backup system. Sorry for not mentioning, I am planning on using XFS. Its intended usage is general home use; probably most of the space will end up being used by media files that would typically be accessed over the network by MythTV boxes. I'll also be using it as a sandbox database/web/mail server. Everything will just be personal stuff, so if the I did lose it all I would be very depressed, but I hopefully will have all the most important stuff backed up, and I won't lose my job or anything too horrible. The main reason I'm concerned about performance is that for some time after I buy it, it will be the highest speced of my boxes, and so I will also be using it for some gaming, which is where I expect performance to be most noticeable. beolach I'm also interested in hearing people's opinions about beolach LVM / EVMS. They are yellow, and taste of vanilla :-). To say something more specific is difficult without knowing what kind of requirement they may be expected to satisfy. beolach I'm currently planning on just using RAID w/out the beolach higher level volume management, as from my reading I beolach don't think they're worth the performance penalty, [ beolach ... ] Very amusing that someone who is planning to grow a 3 drive RAID5 into a 14 drive RAID6 worries about the DM performance penalty. Well, I was reading that LVM2 had a 20%-50% performance penalty, which in my mind is a really big penalty. But I think those numbers where from some time ago, has the situation improved? And is a 14 drive RAID6 going to already have enough overhead that the additional overhead isn't very significant? I'm not sure why you say it's amusing. The other reason I wasn't planning on using LVM was because I was planning on keeping all the drives in the one RAID. If I decide a 14 drive array is too risky, and I go w/ 2 or 3 arrays then LVM would appear much more useful to me. Thanks for the response, Conway S. Smith - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: RAID5 to RAID6 reshape?
Conway S. Smith said: (by the date of Sun, 17 Feb 2008 07:45:26 -0700) Well, I was reading that LVM2 had a 20%-50% performance penalty, huh? Make a benchmark. Do you really think that anyone would be using it if there was any penalty bigger than 1-2% ? (random access, r/w). I have no idea what is the penalty, but I'm totally sure I didn't notice it. -- Janek Kozicki | - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
