On Sat, 2012-03-24 at 01:08 -0700, Anton Cohen wrote: > On Fri, Mar 23, 2012 at 10:01 AM, Conrad Wood <[email protected]> > wrote: > on storage: > dd if=/dev/zero of=volume bs=1M : ~ 1,600MByte/s > cp file1 file2 : ~ 300MByte/s (both files on same volume) > > on server > dd if=/dev/zero of=volume bs=1M : ~ 1,100MByte/s > cp file1 file2 : ~ 30MByte/s (both files on same volume) > > It's not clear to me whether of=volume is writing to the volume or a > file on a file system, i.e., is it of=/dev/vol or > of=/mountpoint/file1? To be a fair test against 'cp' you should have > 'dd' write to a file, though the performance should be fairly close. > Also I can't tell if they are direct/synced writes or not, so I'll > assume not (sorry for assuming). >
quite true, sorry for the ambiquity on my side. of=volume refers to a file on the volume. Die direct option in the linux kernel isn't quite what I wanted, because apparetntly it keeps only a single BIO in-flight at any time. (Which matches my observation) That slows things down a lot of course and isn't what 'cp' does. I issue a 'sync' command afterwards and calculate the amount of bytes written & read between starting "cp" or "dd" and the "sync" being finished. > > Your 'dd' numbers look suspiciously high, like buffered writes to RAM. > Whenever you want to test actual write performance with 'dd', not > memory or dirty page performance, you need to do oflag=direct: > dd if=/dev/zero of=/mnt/point/file oflag=direct bs=1M count=1000 Actually it is a 24-disk raid array with pretty fast disks. I do believe the numbers are correct for sequential(ish) read/writes. (each disk delivers ~120MByte/s sequential, so the raid should theoretically give 2.8GByte/s. (even though the SAS bus cannot go quite as fast I believe) > > > Unless you actually want to test writing dirty pages, and I have seen > a kernel bug where flushing dirty pages was slow, but then you should > also time a 'sync': > time dd if=/dev/zero of=/mnt/point/file bs=1M count=1000 && time sync > Mostly I wonder if it is atall possible to get such speeds over QDR. Are you in a position where you could perhaps run a "cp" followed by "sync" on an infiniband attached storage system? _______________________________________________ Tech mailing list [email protected] https://lists.lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
