Re: RAID-5 streaming read performance

2005-07-15 Thread Ming Zhang
in my previous test, using SATA, i got better result in 2.6 instead of 2.4. :P Ming On Fri, 2005-07-15 at 06:01 +, Holger Kiehl wrote: I'm trying to figure out why the last two numbers differ. Have you checked what the performance with a 2.4.x kernel is? If I remember correctly there

Re: RAID-5 streaming read performance

2005-07-14 Thread Ming Zhang
On Wed, 2005-07-13 at 23:58 -0400, Dan Christensen wrote: David Greaves [EMAIL PROTECTED] writes: In my setup I get component partitions, e.g. /dev/sda7: 39MB/s raid device /dev/md2: 31MB/s lvm device /dev/main/media: 53MB/s (oldish system - but note that

Re: RAID-5 streaming read performance

2005-07-14 Thread Ming Zhang
my problem here. this only apply to sdX not mdX. pls ignore this. ming On Thu, 2005-07-14 at 08:30 -0400, Ming Zhang wrote: Also, is there a way to disable caching of reads? Having to clear the cache by reading 900M each time slows down testing. I guess I could reboot with mem=100M, but

Re: RAID-5 streaming read performance

2005-07-14 Thread Dan Christensen
Mark Hahn [EMAIL PROTECTED] writes: Is there a way for me to simulate readahead in userspace, i.e. can I do lots of sequential asynchronous reads in parallel? there is async IO, but I don't think this is going to help you much. Also, is there a way to disable caching of reads? Having to

Re: RAID-5 streaming read performance

2005-07-14 Thread Mark Hahn
i also want a way to clear part of the whole page cache by file id. :) understandably, kernel developers are don't high-prioritize this sort of not-useful-for-normal-work feature. i also want a way to tell the cache distribution, how many for file A and B, you should probably try

Re: RAID-5 streaming read performance

2005-07-14 Thread Dan Christensen
Ming Zhang [EMAIL PROTECTED] writes: On Thu, 2005-07-14 at 19:29 -0400, Mark Hahn wrote: i also want a way to clear part of the whole page cache by file id. :) understandably, kernel developers are don't high-prioritize this sort of not-useful-for-normal-work feature. agree. Clearing

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Tue, 2005-07-12 at 22:52 -0400, Dan Christensen wrote: Ming Zhang [EMAIL PROTECTED] writes: On Mon, 2005-07-11 at 11:11 -0400, Dan Christensen wrote: I was wondering what I should expect in terms of streaming read performance when using (software) RAID-5 with four SATA drives. I

Re: RAID-5 streaming read performance

2005-07-13 Thread Dan Christensen
Ming Zhang [EMAIL PROTECTED] writes: have u try the parallel write? I haven't tested it as thoroughly, as it brings lvm and the filesystem into the mix. (The disks are in production use, and are fairly full, so I can't do writes directly to the disk partitions/raid device.) My preliminary

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Wed, 2005-07-13 at 08:48 -0400, Dan Christensen wrote: Ming Zhang [EMAIL PROTECTED] writes: have u try the parallel write? I haven't tested it as thoroughly, as it brings lvm and the filesystem into the mix. (The disks are in production use, and are fairly full, so I can't do writes

Re: RAID-5 streaming read performance

2005-07-13 Thread Dan Christensen
Ming Zhang [EMAIL PROTECTED] writes: test on a production environment is too dangerous. :P and many benchmark tool u can not perform as well. Well, I put production in quotes because this is just a home mythtv box. :-) So there are plenty of times when it is idle and I can do benchmarks.

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Wed, 2005-07-13 at 10:23 -0400, Dan Christensen wrote: Ming Zhang [EMAIL PROTECTED] writes: test on a production environment is too dangerous. :P and many benchmark tool u can not perform as well. Well, I put production in quotes because this is just a home mythtv box. :-) So there

Re: RAID-5 streaming read performance

2005-07-13 Thread David Greaves
Dan Christensen wrote: Ming Zhang [EMAIL PROTECTED] writes: test on a production environment is too dangerous. :P and many benchmark tool u can not perform as well. Well, I put production in quotes because this is just a home mythtv box. :-) So there are plenty of times when it is

Re: RAID-5 streaming read performance

2005-07-13 Thread Dan Christensen
Here's a question for people running software raid-5: do you get significantly better read speed from a raid-5 device than from it's component partitions/hard drives, using the simple dd test I did? Knowing this will help determine whether something is funny with my set-up and/or hardware, or if

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Wed, 2005-07-13 at 19:02 +0100, David Greaves wrote: Dan Christensen wrote: Ming Zhang [EMAIL PROTECTED] writes: test on a production environment is too dangerous. :P and many benchmark tool u can not perform as well. Well, I put production in quotes because this is

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Wed, 2005-07-13 at 22:18 +0100, David Greaves wrote: Ming Zhang wrote: component partitions, e.g. /dev/sda7: 39MB/s raid device /dev/md2: 31MB/s lvm device /dev/main/media: 53MB/s (oldish system - but note that lvm device is *much* faster) this is

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Wed, 2005-07-13 at 22:50 +0100, David Greaves wrote: Ming Zhang wrote: On Wed, 2005-07-13 at 22:18 +0100, David Greaves wrote: Ming Zhang wrote: component partitions, e.g. /dev/sda7: 39MB/s raid device /dev/md2: 31MB/s lvm device /dev/main/media:

Re: RAID-5 streaming read performance

2005-07-13 Thread Neil Brown
On Wednesday July 13, [EMAIL PROTECTED] wrote: Here's a question for people running software raid-5: do you get significantly better read speed from a raid-5 device than from it's component partitions/hard drives, using the simple dd test I did? SCSI-160 bus, using just 4 of the 15000rpm

Re: RAID-5 streaming read performance

2005-07-13 Thread Neil Brown
On Wednesday July 13, [EMAIL PROTECTED] wrote: Question for the list: if I'm doing a long sequential write, naively each parity block will get recalculated and rewritten several times, once for each non-parity block in the stripe. Does the write-caching that the kernel does mean that each

Re: RAID-5 streaming read performance

2005-07-13 Thread Neil Brown
On Wednesday July 13, [EMAIL PROTECTED] wrote: I guess each raw device does some readahead, then the md0 does some readahead and then the lvm does some readahead. Theoretically the md0 and lvm should overlap - but I guess that much of the raw device level readahead is discarded. No. Devices

Re: RAID-5 streaming read performance

2005-07-13 Thread Neil Brown
On Wednesday July 13, [EMAIL PROTECTED] wrote: On Thu, 2005-07-14 at 08:38 +1000, Neil Brown wrote: On Wednesday July 13, [EMAIL PROTECTED] wrote: Here's a question for people running software raid-5: do you get significantly better read speed from a raid-5 device than from it's

Re: RAID-5 streaming read performance

2005-07-13 Thread Ming Zhang
On Thu, 2005-07-14 at 11:16 +1000, Neil Brown wrote: On Wednesday July 13, [EMAIL PROTECTED] wrote: On Thu, 2005-07-14 at 08:38 +1000, Neil Brown wrote: On Wednesday July 13, [EMAIL PROTECTED] wrote: Here's a question for people running software raid-5: do you get significantly

Re: RAID-5 streaming read performance

2005-07-13 Thread Dan Christensen
David Greaves [EMAIL PROTECTED] writes: In my setup I get component partitions, e.g. /dev/sda7: 39MB/s raid device /dev/md2: 31MB/s lvm device /dev/main/media: 53MB/s (oldish system - but note that lvm device is *much* faster) Did you test component device and

Re: RAID-5 streaming read performance

2005-07-13 Thread Mark Hahn
component partitions, e.g. /dev/sda7: 39MB/s raid device /dev/md2: 31MB/s lvm device /dev/main/media: 53MB/s (oldish system - but note that lvm device is *much* faster) Did you test component device and raid device speed using the read-ahead settings tuned

Re: RAID-5 streaming read performance

2005-07-12 Thread Ming Zhang
On Mon, 2005-07-11 at 11:11 -0400, Dan Christensen wrote: I was wondering what I should expect in terms of streaming read performance when using (software) RAID-5 with four SATA drives. I thought I would get a noticeable improvement compared to reads from a single device, but that's not the