Pete Wyckoff wrote:
[EMAIL PROTECTED] wrote on Fri, 29 Dec 2006 13:35 -0500:
Do these require the kernel module? I have not tried using that yet.
Nope, but you need to figure out MPI.
There are some BMI tests in test/io/bmi that can be used just to
look at that layer. They do work, but may need some tweaking for
your new device. I haven't taken numbers with them for a long time
though.
I do not see any improvement using larger than 1 MB with
FlowBufferSizeBytes.
Odd. I figured most of the overhead here was server request
processing and that you would also see an improvement.
We've seen similar lack of improvement above 1MB flowbuffers on IB here,
at least for tmpfs.
I was not swapping (I have 8 GB available). Using ramfs instead of
tmpfs, I can get 1,200 MB/s. I have switched to ramfs but the numbers
are roughly the same.
Much better. Wonder how your kernel managed to have a bad tmpfs.
BMI_OPTIMISTIC_BUFFER_REG.
What does this do? Internally, MX can cache some registrations (the
API does not expose it). It is in my best interest to try to reuse
buffers.
Just chunks up the memory registrations. If you don't have an MX API
knob for it, you'll never be able to use it. Very handy for IB to
amortize registration overhead though.
Let us know if you manage to figure out why you're not getting good
MX performance. Perhaps these other tests can shed some light. You
might look at detailed server or client logs with usec timestamps to
figure out where the time is going. And oprofile may be useful if
you think the time is going to the MX libs or driver, or pthread
overhead.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
!DSPAM:45956664175553366512726!
--
Kyle Schochenmaier
[EMAIL PROTECTED]
Research Assistant, Dr. Brett Bode
AmesLab - US Dept.Energy
Scalable Computing Laboratory
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
