[EMAIL PROTECTED] wrote on Mon, 21 Aug 2006 19:51 -0400:
> Sam and Murali brought this up based on my questions about
> BMI_memalloc() and BMI_memfree(). Unlike GM, MX does not have a
> memory registration API. It handles registration within the library
> automatically. The user always passes in plain malloc'd buffers for
> both send and receive. Since the unexpected buffer for a MX BMI would
> simply be a malloc'd buffer, I did not think it made much sense to
> malloc another buffer, copy the data and pass that back to BMI.
>
> Given that, I could live with BMI_memalloc() and BMI_memfree() that
> are simply malloc() and free().
That's just fine. A perfectly reasonable way to implement
BMI_memalloc and BMI_memfree for MX. (Although you may want to
investigate how much it costs to register a buffer under the covers.
It's hopefully only slightly more expensive than free. Hopefully
not worth your time to do memory registration caching like in the IB
case.)
The question about unexpected buffers was a bit more subtle. Some
networks must used registered memory (as you know; I'm just
recapping). For these it's likely faster to reuse a set of
preallocated buffers for unexpected messages. Hence the memcpy to
move data from the "special" buffers into normal memory before
handing it back to the callers, so we can reuse the good memory for
network purposes.
I guess Sam just noticed that this memcpy is not necessary if you
are willing to add an acknowledgment protocol between BMI client and
BMI network. Definitely a good suggestion. Even for networks that
don't need special memory, it's still worth doing to avoid the
library-malloc but user-free asymmetry that Sam pointed out.
-- Pete
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
