On Feb 3, 2005, at 9:40 AM, Nuno Silva wrote:
Ian Godin wrote:I am trying to get very fast disk drive performance and I am seeing some interesting bottlenecks. We are trying to get 800 MB/sec or more (yes, that is megabytes per second). We are currently using PCI-Express with a 16 drive raid card (SATA drives). We have achieved that speed, but only through the SG (SCSI generic) driver. This is running the stock 2.6.10 kernel. And the device is not mounted as a file system. I also set the read ahead size on the device to 16KB (which speeds things up a lot):
I was trying to reproduce but got distracted by this: (use page down, if you just want to see the odd result)
puma:/tmp/dd# sg_map /dev/sg0 /dev/sda /dev/sg1 /dev/sdb /dev/sg2 /dev/scd0 /dev/sg3 /dev/sdc puma:/tmp/dd# time sg_dd if=/dev/sg1 of=/tmp/dd/sg1 bs=64k count=1000 Reducing read to 64 blocks per loop 1000+0 records in 1000+0 records out
real 0m0.187s user 0m0.001s sys 0m0.141s puma:/tmp/dd# time dd if=/dev/sdb of=/tmp/dd/sdb bs=64k count=1000 1000+0 records in 1000+0 records out 65536000 bytes transferred in 1.203468 seconds (54455956 bytes/sec)
real 0m1.219s user 0m0.001s sys 0m0.138s puma:/tmp/dd# ls -l total 128000 -rw-r--r-- 1 root root 65536000 Feb 3 17:16 sdb -rw-r--r-- 1 root root 65536000 Feb 3 17:16 sg1 puma:/tmp/dd# md5sum * ec31224970ddd3fb74501c8e68327e7b sdb 60d4689227d60e6122f1ffe0ec1b2ad7 sg1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
See? dd from sdb is not the same as sg1! Is this supposed to happen?
About the 900MB/sec:
This same sg1 (= sdb, which is a single hitachi sata hdd) performes like this:
puma:/tmp/dd# time sg_dd if=/dev/sg1 of=/dev/null bs=64k count=1000000 time=1
Reducing read to 64 blocks per loop
time to transfer data was 69.784784 secs, 939.12 MB/sec
1000000+0 records in
1000000+0 records out
real 1m9.787s user 0m0.063s sys 0m58.115s
I can assure you that this drive can't do more than 60MB/sec sustained.
My only conclusion is that sg (or sg_dd) is broken? ;)
Peace, Nuno Silva
Definitely have been able to repeat that here, so the SG driver definitely appears to be broken. At least I'm glad I am not going insane, I was starting to wonder :)
I'll run some more tests with O_DIRECT and such things, see if I can figure out what the REAL max speed is.
Thanks for the help everyone, Ian.
- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
