[EMAIL PROTECTED] wrote on Mon, 27 Feb 2006 14:38 -0600: > I've been working with the 1.4.0 release and got everything up and > running, running tests, etc. > Currently I have only one bug which pops up randomly and causes the > server to kill itself unexpectedly, but I'll hopefully find that soon.... > So far I've managed to see a max rate of about 180MB/s from pvfs2-cp. > I havent really seen any single-client data for the vapi-Infiniband stuff. > How are these results looking? > - Does anyone have a link to, or data for tests similar to mine over > the VAPI IB stack? (single-client && single-server)
Your numbers look great. There never was a paper on pvfs2 on IB, so no definitive resource, but there is the old pvfs1 on IB one. (Grep "PVFS over InfiniBand" in http://www.osc.edu/~pw/papers/.) That old paper shows 1-1 numbers of 400 MB/s for read or write, but it's really not fair to compare against that. I just tested vapi with some tmpfs on a couple of opteron machines, like your setup, and got 180 MB/s write, 120 MB/s read using pvfs2-cp with IB/vapi. So you're doing just fine as far as I'm concerned. The multi-client and multi-server benchmarks will be interesting to see, although you'll need to use an openib mpich. If you try the OSU mvapiches, you will be in an interesting situation. mvapich-gen2 is based on old mpich1 code with ROMIO that needs to be patched to work with pvfs2. mvapich2-gen2 isn't available on the OSU website, but it is in the openib distro. I don't think it has gotten much testing... I can help you off-list with mvapich1 romio patching. -- Pete _______________________________________________ Pvfs2-developers mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
