On Nov 30, 2006, at 9:27 AM, Scott Atchley wrote:
Hi all,
No one replied the the original post. The one I am most curious
about is (3). Is FlowBufferSizeBytes available to the BMI
implementations?
Hi Scott,
Sorry for not responding. I've included responses to your questions
inline.
I am thinking about how to handle BMI_mx_memalloc(). I might be
able to assist the reg cache in MX if I do things a certain way.
I mention this below, on the server we always call BMI_memalloc to
allocate buffers and post them to BMI_post_send/BMI_post_recv. On
the client that's not always the case, the user buffer is usually
passed directly to BMI_post_send/BMI_post_recv. Would it help you to
know in advance of the post (or all posts) what the maximum size of a
buffer for BMI_post_send/BMI_post_recv is going to be? We might be
able to add a BMI info option that other methods could ignore. We do
limit it, but its all above the BMI interface that this is done.
Scott
On Oct 25, 2006, at 1:53 PM, Scott Atchley wrote:
Hi all,
A few quick questions:
1. Is there an upper bound on how many transfer operations between
a pair of hosts at any one time (i.e. between a client and host)?
There is a max of unexpected requests (UnexpectedRequests option in
the config file, defaults to 50). This limits the number of
unexpected messages that can be handled at once by the server, but
doesn't limit the number of IO operations (large expected messages).
2. If there is a limit in the above and it is greater than 1, is
that value the same for small (unexpected and some expected)
messages and large (expected) messages? Or are large messages
restricted to a lower value?
So I don't think there's a limit at present for expected messages.
The unexpected limit is the one I mentioned above.
3. In the thread entitled "libpvfs2 usage", SamL mentioned the
tunable called FlowBufferSizeBytes and that it is set to 256KB by
default. Is this value the upper limit on large messages (i.e. 8KB
< large messages <= FlowBufferSizeBytes assuming 8 KB is the
maximum unexpected size)?
Yes, on the server BMI_post_recv and BMI_post_send won't get called
with values larger than that. On the client, there's no strict limit
on the size of the messages posted.
Would the BMI method have any knowledge of what this value is
(i.e. can it use FlowBufferSizeBytes to allocate large buffers)?
On the server it does. BMI_memalloc is called from the flow with
exactly the 256K (FlowBufferSizeBytes) value. The buffer returned is
the one passed to BMI_post_recv or BMI_post_send (although, the size
of the request may be less than 256K). On the client, the user
buffer is used for calls to BMI_post_recv and BMI_post_send, so you
won't see BMI_memalloc calls first. The client does break up the
entire client request into chunks of FlowBufferSizeBytes for the BMI
posts, but you won't know what that value is in advance of the post
operations.
-sam
Thanks,
Scott
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
_______________________________________________
Pvfs2-developers mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
