400MB/s is what I commonly saw when using the 20Gbit DDR nics between a single client and single server... I dont think I've seen much faster in that setup, regardless of hardware.
Now the real fun will be getting that 400MB/s onto disk :-) Good luck Kyle On 9/6/07, Salmon, Rene <[EMAIL PROTECTED]> wrote: > > Hi, > > > I got the latest pvfs from CVS and rebuilt both the server and the > client with the "--without-bmi-tcp" flag. This took me from 260MB/sec > to about 320MB/sec with pvfs2-cp. About a 60MB/sec boost. > > adjusting the buffer size option to pvfs2-cp did help: > > pvfs2-cp -b16777216 -t /tmp/testfile1 /test/pvfs2/foo12 > Wrote 16777216000 bytes in 45.138288 seconds. 354.466257 MB/seconds > > pvfs2-cp -b33554432 -t /tmp/testfile1 /test/pvfs2/foo13 > Wrote 16777216000 bytes in 43.276829 seconds. 369.712855 MB/seconds > > pvfs2-cp -b67108864 -t /tmp/testfile1 /test/pvfs2/foo14 > Wrote 16777216000 bytes in 40.825193 seconds. 391.914866 MB/seconds > > pvfs2-cp -b104857600 -t /tmp/testfile1 /test/pvfs2/foo17 > Wrote 16777216000 bytes in 39.127303 seconds. 408.921615 MB/seconds > > pvfs2-cp -b134217728 -t /tmp/testfile1 /test/pvfs2/foo15 > Wrote 16777216000 bytes in 41.528605 seconds. 385.276606 MB/seconds > > > Best I can get is around 400MB/s for a one(client) to one (server) test > config. This is really close to what my disk back end can handle. > Since there is only one I/O server using the -s option for striping did > not help any in this case. > > Thanks > Rene > > > > On Thu, 2007-09-06 at 19:16 +0100, Scott Atchley wrote: > > Hi Rene, > > > > First, what is your FlowBufferSize set to in your fs.conf file? > > > > Second, I found that by playing with pvfs2-cp's switches, I could > > improve performance some. I typically use a FlowBufferSize of 4 MB. > > Pete would have to comment on what he uses with IB. Given that, I set > > -b $((N*1024*1024)) where N is 16, 20, 32, 40, 64, 100, and 128. You > > may also want to try using -s $((M*$FlowBufferSize)) where M is 1, 2, > > 4, etc. > > > > Scott > > > > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > -- Kyle Schochenmaier
_______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
