Re: striping of a 4 drive raid10
Keld Jørn Simonsen wrote: On Mon, Jan 28, 2008 at 07:13:30AM +1100, Neil Brown wrote: On Sunday January 27, [EMAIL PROTECTED] wrote: Hi I have tried to make a striping raid out of my new 4 x 1 TB SATA-2 disks. I tried raid10,f2 in several ways: 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 of md0+md1 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 of md0+md1 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of md0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize of md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 Try 6: md0 = raid10,f2 of sda1+sdb1+sdc1+sdd1 That I already tried, (and I wrongly stated that I used f4 in stead of f2). I had two times a thruput of about 300 MB/s but since then I could not reproduce the behaviour. Are there errors on this that has been corrected in newer kernels? Also try raid10,o2 with a largeish chunksize (256KB is probably big enough). I tried that too, but my mdadm did not allow me to use the o flag. My kernel is 2.6.12 and mdadm is v1.12.0 - 14 June 2005. can I upgrade the mdadm alone to a newer version, and then which is recommendable? I doubt that updating the mdadm is going to help, the kernel is old and lacks a number of improvements in the last few years. I don't think you will see any major improvements without a kernel upgrade. -- Bill Davidsen [EMAIL PROTECTED] Woe unto the statesman who makes war without a reason that will still be valid when the war is over... Otto von Bismark - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: striping of a 4 drive raid10
On Mon, Jan 28, 2008 at 01:32:48PM -0500, Bill Davidsen wrote: Neil Brown wrote: On Sunday January 27, [EMAIL PROTECTED] wrote: Hi I have tried to make a striping raid out of my new 4 x 1 TB SATA-2 disks. I tried raid10,f2 in several ways: 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 of md0+md1 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 of md0+md1 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of md0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize of md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 Try 6: md0 = raid10,f2 of sda1+sdb1+sdc1+sdd1 Also try raid10,o2 with a largeish chunksize (256KB is probably big enough). Looking at the issues raised, there might be some benefit from having the mirror chunks on the slower inner tracks of a raid10, and to read from the outer tracks if the drives with the data on the outer tracks are idle. This would appear to offer a transfer rate benefit overall. Hmm, how do I do this? I think this is normal behaviour of a raid10,f2. Is that so? So you mean I should rather use f2 than o2? Or should I configure the f2 in some way? My hdparm -t gives: /dev/sda5: Timing buffered beginning disk reads: 82 MB in 1.00 seconds = 81.686 MB/sec Timing buffered endingdisk reads: 42 MB in 1.03 seconds = 40.625 MB/sec Average seek time 13.714 msec, min=4.641, max=23.921 Average track-to-track time 28.151 msec, min=26.729, max=28.730 So, yes, there is a reason to use the faster outer tracks - and have the faster access time that f2 gives . How does o2 behave here? Does it read and search on the whole disk? As to your other comments in another mail, I could of cause install a newer kernel and mdadm, but then I would loose the support of my supported and paid system. And Neil said that there have been no performance fixes for f2 since the kernel I use (2.6.12). I thought that o2 support was included since 2.6.10 - but apparantly not so. Best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
striping of a 4 drive raid10
Hi I have tried to make a striping raid out of my new 4 x 1 TB SATA-2 disks. I tried raid10,f2 in several ways: 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 of md0+md1 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 of md0+md1 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of md0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize of md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 My new disks give a transfer rate of about 80 MB/s, so I expected to have something like 320 MB/s for the whole raid, but I did not get more than about 180 MB/s. I think it may be something with the layout, that in effect the drives should be something like: sda1 sdb1sdc1 sdd1 01 2 3 45 6 7 And this was not really doable for the combination of raids, because thet combinations give different block layouts. How can it be done? Do we need a new raid type? Best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: striping of a 4 drive raid10
On Sun, 27 Jan 2008 20:33:45 +0100, Keld Jørn Simonsen [EMAIL PROTECTED] said: keld Hi I have tried to make a striping raid out of my new 4 x keld 1 TB SATA-2 disks. I tried raid10,f2 in several ways: keld 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 keldof md0+md1 keld 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 keldof md0+md1 keld 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of keldmd0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB keld 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize keldof md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB These stacked RAID levels don't make a lot of sense. keld 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 This also does not make a lot of sense. Why have four mirrors instead of two? Instead, try 'md0 = raid10,f2' for example. The first mirror of will be striped across the outer half of all four drives, and the second mirrors will be rotated in the inner half of each drive. Which of course means that reads will be quite quick, but writes and degraded operation will be slower. Consider this post for more details: http://www.spinics.net/lists/raid/msg18130.html [ ... ] - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: striping of a 4 drive raid10
On Sunday January 27, [EMAIL PROTECTED] wrote: Hi I have tried to make a striping raid out of my new 4 x 1 TB SATA-2 disks. I tried raid10,f2 in several ways: 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 of md0+md1 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 of md0+md1 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of md0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize of md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 Try 6: md0 = raid10,f2 of sda1+sdb1+sdc1+sdd1 Also try raid10,o2 with a largeish chunksize (256KB is probably big enough). NeilBrown My new disks give a transfer rate of about 80 MB/s, so I expected to have something like 320 MB/s for the whole raid, but I did not get more than about 180 MB/s. I think it may be something with the layout, that in effect the drives should be something like: sda1 sdb1sdc1 sdd1 01 2 3 45 6 7 And this was not really doable for the combination of raids, because thet combinations give different block layouts. How can it be done? Do we need a new raid type? Best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: striping of a 4 drive raid10
On Mon, Jan 28, 2008 at 07:13:30AM +1100, Neil Brown wrote: On Sunday January 27, [EMAIL PROTECTED] wrote: Hi I have tried to make a striping raid out of my new 4 x 1 TB SATA-2 disks. I tried raid10,f2 in several ways: 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 of md0+md1 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 of md0+md1 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of md0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize of md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 Try 6: md0 = raid10,f2 of sda1+sdb1+sdc1+sdd1 That I already tried, (and I wrongly stated that I used f4 in stead of f2). I had two times a thruput of about 300 MB/s but since then I could not reproduce the behaviour. Are there errors on this that has been corrected in newer kernels? Also try raid10,o2 with a largeish chunksize (256KB is probably big enough). I tried that too, but my mdadm did not allow me to use the o flag. My kernel is 2.6.12 and mdadm is v1.12.0 - 14 June 2005. can I upgrade the mdadm alone to a newer version, and then which is recommendable? best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: striping of a 4 drive raid10
On Sun, Jan 27, 2008 at 08:11:35PM +, Peter Grandi wrote: On Sun, 27 Jan 2008 20:33:45 +0100, Keld Jørn Simonsen [EMAIL PROTECTED] said: keld Hi I have tried to make a striping raid out of my new 4 x keld 1 TB SATA-2 disks. I tried raid10,f2 in several ways: keld 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 keldof md0+md1 keld 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 keldof md0+md1 keld 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of keldmd0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB keld 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize keldof md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB These stacked RAID levels don't make a lot of sense. keld 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 This also does not make a lot of sense. Why have four mirrors instead of two? My error, I did mean f2. Anyway 4 mirrors would make the disk 2 times faster than 2 disks, and given disk prices these days this could make a lot of sense. Instead, try 'md0 = raid10,f2' for example. The first mirror of will be striped across the outer half of all four drives, and the second mirrors will be rotated in the inner half of each drive. Which of course means that reads will be quite quick, but writes and degraded operation will be slower. Consider this post for more details: http://www.spinics.net/lists/raid/msg18130.html Thanks for the reference. There is also more in the original article on possible layouts of what is now known as raid10,f2 http://marc.info/?l=linux-raidm=107427614604701w=2 including performance enhancements due to use of the faster outer sectors, and smaller average seek times because you can seek on only half the disk. best regards keld - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: striping of a 4 drive raid10
On Sunday January 27, [EMAIL PROTECTED] wrote: On Mon, Jan 28, 2008 at 07:13:30AM +1100, Neil Brown wrote: On Sunday January 27, [EMAIL PROTECTED] wrote: Hi I have tried to make a striping raid out of my new 4 x 1 TB SATA-2 disks. I tried raid10,f2 in several ways: 1: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, md2 = raid0 of md0+md1 2: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, md2 = raid01,f2 of md0+md1 3: md0 = raid10,f2 of sda1+sdb1, md1= raid10,f2 of sdc1+sdd1, chunksize of md0 =md1 =128 KB, md2 = raid0 of md0+md1 chunksize = 256 KB 4: md0 = raid0 of sda1+sdb1, md1= raid0 of sdc1+sdd1, chunksize of md0 = md1 = 128 KB, md2 = raid01,f2 of md0+md1 chunksize = 256 KB 5: md0= raid10,f4 of sda1+sdb1+sdc1+sdd1 Try 6: md0 = raid10,f2 of sda1+sdb1+sdc1+sdd1 That I already tried, (and I wrongly stated that I used f4 in stead of f2). I had two times a thruput of about 300 MB/s but since then I could not reproduce the behaviour. Are there errors on this that has been corrected in newer kernels? No, I don't think any performance related changes have been made to raid10 lately. You could try increasing the read-ahead size. For a 4-drive raid10 it defaults to 4 times the read-ahead setting of a single drive, but increasing substantially (e.g. 64 times) seem to increase the speed of dd reading a gigabyte. Whether that will actually affect your target workload is a different question. Also try raid10,o2 with a largeish chunksize (256KB is probably big enough). I tried that too, but my mdadm did not allow me to use the o flag. My kernel is 2.6.12 and mdadm is v1.12.0 - 14 June 2005. can I upgrade the mdadm alone to a newer version, and then which is recommendable? You would need a newer kernel and a newer mdadm to get raid10 - offset mode. NeilBrown - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
