Re: Help: very slow software RAID 5.
One of the 5-10 tuning settings: blockdev --getra /dev/md0 Try setting it to 4096,8192,16384,32768,65536 blockdev --setra 4096 /dev/md0 Also, with a 3-disk raid5 that is the worst performance you can get using only 3 disks, while with a 10 disk raid5 it'd be closer to 90%. Reads you should get good speeds, but for writes-- probably not. Then re-benchmark. Justin. On Tue, 18 Sep 2007, Dean S. Messing wrote: Justin Piszcz wrote: On Tue, 18 Sep 2007, Dean S. Messing wrote: : : : : I'm not getting nearly the read speed I expected : from a newly defined software RAID 5 array : across three disk partitions (on the 3 drives, : of course!). : : Would someone kindly point me straight? : : After defining the RAID 5 I did `hdparm -t /dev/md0' : and got the abysmal read speed of ~65MB/sec. : The individual device speeds are ~55, ~70, : and ~75 MB/sec. : : Shouldn't this array be running (at the slowest) : at about 55+70 = 125 MB/sec minus some overhead? : I defined a RAID0 on the ~55 and ~70 partitions : and got about 110 MB/sec. : : Shouldn't adding a 3rd (faster!) drive into the : array make the RAID 5 speed at least this fast? : : : Here are the details of my setup: : snip : Without tuning you will get very slow read speed. : : Read the mailing list, there are about 5-10 tunable options, for me, I get : 250 MiB/s no tuning (read/write), after tuning 464 MiB/s write and 622 : MiB/s read. : : Justin. Thanks Justin. 5-10 tunable options! Good grief. This sounds worse than regulating my 80 year old Grandfather Clock. (I'm quite a n00bie at this.) Are there any nasty system side effects to tuning these parameters? This is not a server I'm working with. It's my research desktop machine. I do lots and lots of different things on it. I started out with RAID 0, but after reading a lot I learned that this is Dangerous. So I bought a 3rd disk to do RAID 5. I intend to put LVM on top of the RAID 5 once I get it running at the speed it is supposed to, and then copy my entire linux system onto it. Are these tuned parameters going to mess other things up? Is there official documentation on this relative to RAID 5? I don't see much online. One thing I did learn is that if I use the --direct switch with `hdparm' I get much greater read speed. Goes from ~65 MB/s to ~120 MB/s. I have no idea what --direct does. Yes, I've read the man page. It says that it causes bypassing of the page cache. Ooookay. Alas using dd I find that the real read speed is still around ~65 MB/s. Sorry for these musings. I'm just uncomfortable trying to diddle with 5-10 system parameters without knowing what I'm doing. Any help or pointers to documentaion on tuning these for RAID-5 and what the tradeoffs are would be appreciated. Thanks. Dean - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
md device naming question
hello, working on initramfs i'd be curious to know what the /sys/block entry of a /dev/md/NN device is. have a user request to support it and no handy box using it. i presume it may also be /sys/block/mdNN ? thanks -- maks - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Help: very slow software RAID 5.
Justin Piszcz wrote: : One of the 5-10 tuning settings: : : blockdev --getra /dev/md0 : : Try setting it to 4096,8192,16384,32768,65536 : : blockdev --setra 4096 /dev/md0 : : I discovered your January correspondence to the list about this. Yes, the read-ahead length makes a dramtic difference---for sequential data reading. However, in doing some further study on this parameter, I see that random access is going to suffer. Since I had intended to build an LVM on this RAID 5 array and put a full linux system on it, I'm not sure that large read-ahead values are a good idea. : Also, with a 3-disk raid5 that is the worst performance you can get using : only 3 disks I don't know why (for reads) I should suffer such bad performance. According to all I've read, the system is not even supposed to read the parity data on reads. So why do I not get near RAID 0 speeds w/o having to increase the Read-ahead value? : , while with a 10 disk raid5 it'd be closer to 90%. Reads you : should get good speeds, but for writes-- probably not. When you say reads you should get good speeds are you referring to the aforementioned 10 disk RAID 5 or my 3 disk one? : Then re-benchmark. Large Read-ahead nearly double the speed of 'hdparm -t'. (So does simply using the --direct flag. Can you explain this?) Also, in your opinion, is it wise to use such large read-aheads for a RAID 5 intended for the use to which I plan to put it? Aside: I have found RAID quite frustrating. With the original two disks I was getting 120-130 MB/s in RAID 0. I would think that for the investment of a 3rd drive I ought to get the modicum of redundancey I expect and keep the speed (at least on reads) w/o sacrificing anything. But it appears I actually lost something for my investment. I'm back to the speed of single drives with the modicum of redundancey that RAID 5 gives. Not a very good deal. Dean - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Help: very slow software RAID 5.
On Wed, 19 Sep 2007, Dean S. Messing wrote: Justin Piszcz wrote: : One of the 5-10 tuning settings: : : blockdev --getra /dev/md0 : : Try setting it to 4096,8192,16384,32768,65536 : : blockdev --setra 4096 /dev/md0 : : I discovered your January correspondence to the list about this. Yes, the read-ahead length makes a dramtic difference---for sequential data reading. However, in doing some further study on this parameter, I see that random access is going to suffer. Since I had intended to build an LVM on this RAID 5 array and put a full linux system on it, I'm not sure that large read-ahead values are a good idea. : Also, with a 3-disk raid5 that is the worst performance you can get using : only 3 disks I don't know why (for reads) I should suffer such bad performance. According to all I've read, the system is not even supposed to read the parity data on reads. So why do I not get near RAID 0 speeds w/o having to increase the Read-ahead value? : , while with a 10 disk raid5 it'd be closer to 90%. Reads you : should get good speeds, but for writes-- probably not. When you say reads you should get good speeds are you referring to the aforementioned 10 disk RAID 5 or my 3 disk one? : Then re-benchmark. Large Read-ahead nearly double the speed of 'hdparm -t'. (So does simply using the --direct flag. Can you explain this?) Also, in your opinion, is it wise to use such large read-aheads for a RAID 5 intended for the use to which I plan to put it? Aside: I have found RAID quite frustrating. With the original two disks I was getting 120-130 MB/s in RAID 0. I would think that for the investment of a 3rd drive I ought to get the modicum of redundancey I expect and keep the speed (at least on reads) w/o sacrificing anything. But it appears I actually lost something for my investment. I'm back to the speed of single drives with the modicum of redundancey that RAID 5 gives. Not a very good deal. Dean - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Did you try increasing the readahead and benchmarking? You can set it back to what it was when you are done. Justin. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: mke2fs stuck in D state while creating filesystem on md*
On 9/19/07, Wiesner Thomas [EMAIL PROTECTED] wrote: Has there been any progress on this? I think I saw it, or something similar, during some testing of recent 2.6.23-rc kernels, on mke2fs took about 11 min longer than all the others (~2 min) and it was not repeatable. I worry that process of more interest will have the same hang. Well, I must say: no. I haven't tried anything further. I've set up the production system a week or so ago which runs Debian Etch with no modifications (kernel 2.6.18 I think, the debian one and a mdadm 2.5.6-9). I didn't notice the problem while creating the raid but that doesn't mean anything as I didn't pay attention and as I wrote earlier it isn't reliably reproducable. (Watching it on a large storage gets boring very fast.) I'm not a kernel programmer but I can test another kernel or mdadm version if it helps, but let me know if you want me to do that. If/when you experience the hang again please get a trace of all processes with: echo t /proc/sysrq-trigger Of particular interest is the mke2fs trace; as well as any md threads. - To unsubscribe from this list: send the line unsubscribe linux-raid in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
