Hi, Yeah you are right. I setup a tmpfs file system (/scratch) in RAM on my test client and here is what I got. Sorry the local SATA disk on the client is not fast enough to keep up so I have to use RAM.
Read from pvfs2 write to RAM tmpfs (/scratch) ---------------------------------------------- hpca4000(salmr0)177:pvfs2-cp -t -b 4194304 /pvfs2-fs/salmr0/Hmig-8GB /scratch/ Wrote 8590895600 bytes in 18.850570 seconds. 434.624338 MB/seconds Write to pvfs2 read from RAM tmpfs (/scratch) --------------------------------------------- hpca4000(salmr0)179:pvfs2-cp -t -b 4194304 /scratch/Hmig-8GB /pvfs2-fs/salmr0/Hmig-8GB-junk Wrote 8590895600 bytes in 13.541110 seconds. 605.040242 MB/seconds Read from pvfs2 write to pvfs2 ------------------------------ hpca4000(salmr0)182:pvfs2-cp -t -b 4194304 /pvfs2-fs/salmr0/Hmig-8GB /pvfs2-fs/salmr0/Hmig-8GB-junk2 Wrote 8590895600 bytes in 21.963378 seconds. 373.026249 MB/seconds I will give the patched pvfs2-cp a try. Thanks Rene > -----Original Message----- > From: Sam Lang [mailto:[EMAIL PROTECTED] > Sent: Wednesday, October 31, 2007 3:20 PM > To: Salmon, Rene > Cc: Scott Atchley; pvfs2-users > Subject: Re: [Pvfs2-users] Pvfs performance over 10Ge > > > On Oct 31, 2007, at 2:58 PM, Salmon, Rene wrote: > > > Hi Sam, > > > > Thanks for the reply. You where right. Taking out all the > IB stuff > > made a big difference. Now I am seeing much better > performance using > > bmi_tcp. Thanks. > > Hi Rene, > > Good to hear it worked out. The configure script will warn > about enabling multiple BMI methods at once, although, I > think it actually assumes tcp is the slow one, whereas in > your case its IB. > > > > > hpca4000(salmr0)140:dd if=/dev/zero of=/pvfs2-fs/salmr0/foo1 bs=4M > > count=2000 > > 2000+0 records in > > 2000+0 records out > > 8388608000 bytes (8.4 GB) copied, 21.3792 seconds, 392 MB/s > > > > hpca4000(salmr0)137:pvfs2-cp -b4194304 -t /pvfs2-fs/salmr0/Hmig-8GB > > /pvfs2-fs/salmr0/Hmig-8GB-copy Wrote 8590895600 bytes in 50.296533 > > seconds. 162.892271 MB/seconds > > > > hpca4000(salmr0)142:pvfs2-cp -t -b 4194304 /vol0/salmr0/Hmig-8GB > > /pvfs2-fs/salmr0/foo-4 > > Wrote 8590895600 bytes in 14.417761 seconds. 568.251657 MB/seconds > > That's interesting, I would have expected the second test (write to > pvfs) to be about twice as fast as the first test (read from > pvfs, write to pvfs). You're seeing writes that are much > faster than reads. What numbers do you get copying a file > from pvfs to local? > > Also, pvfs2-cp is a little inefficient because it does > blocking reads and writes using a single buffer. I've > attached a patch to pvfs2-cp that uses multiple buffers. If > you're inclined, you might see even better numbers with it. > > -sam > > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
