An additional thing to note, the latest versions of coreutils 6.x+ uses larger transfer sizes (64KB?) and should perform significantly better.
You should be able to both see similar performance to cp by using #1, and get an idea of what an upgraded coreutils version of cp might yield : #1 dd if=/local/file of=/mnt/pvfs2/file bs=1024 #2 dd if=/local/file of=/mnt/pvfs2/file bs=65536 I think Phil mentioned this last year : http://www.beowulf-underground.org/pipermail/pvfs2-users/2009-May/002805.html ~Kyle Kyle Schochenmaier On Fri, Apr 16, 2010 at 10:16 AM, Phil Carns <[email protected]> wrote: > Others have already answered with good information, but I also wanted to > mention that cp is slow for PVFS on some systems in part due to using buffer > sizes that are too small. You might get better command line performance > using "dd bs=4M ..." instead of "cp" to copy data from /tmp to /mnt/pvfs2. > The dd that you are using for /tmp works fine with a small transfer size of > 1024 because of caching effects combined with low latency to local storage. > > The lack of client-side caching in PVFS works against it on serial > benchmarks, but it comes back around to being a positive on large parallel > benchmarks. > > -Phil > > On 04/16/2010 09:25 AM, Christian Baun wrote: >> >> Hi, >> >> I created 2 Instances. 1 Instance has an EBS-Volume attached. The >> EBS-Volume is the storage for the PVFS2. Both instances (m1.large) are >> inside EC2 us-east-1a. >> >> The performance looks quite well for large files: >> >> # mount | grep pvfs >> /dev/sdc on /pvfs2-storage-space type ext3 (rw) >> tcp://domU-12-31-38-00-4A-42:3334/pvfs2-fs on /mnt/pvfs2 type pvfs2 (rw) >> >> # dd if=/dev/zero of=/tmp/testfile bs=1024 count=1000000 >> 1000000+0 records in >> 1000000+0 records out >> 1024000000 bytes (1.0 GB) copied, 6.83677 s, 150 MB/s >> >> # pvfs2-cp -t /tmp/testfile /mnt/pvfs2/ >> Wrote 1024000000 bytes in 24.781117 seconds. 39.407525 MB/seconds >> >> When using the PVFS2 Linux Kernel Interface it need much longer: >> >> # time cp /tmp/testfile /mnt/pvfs2/test >> real 2m2.695s >> user 0m0.000s >> sys 0m0.020s >> >> Any ideas why it needs so much longer when using the Linux Kernel >> Interface and how to benchmark the performance in a better way? >> >> :-) >> >> Best Regards, >> Christian >> >> >> >> >> Am Donnerstag, 15. April 2010 schrieb Emmanuel Florac: >> >>> >>> Le Thu, 15 Apr 2010 22:51:00 +0200 vous écriviez: >>> >>> >>>> >>>> Thanks a lot!!! >>>> Now the module is built. :-) >>>> >>> >>> Good luck :) I'm interested in knowing how pvfs performs on EC2... >>> >>> >> >> >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
