[AMD Official Use Only - Internal Distribution Only]
Dear George,
Thank you the reply. But my question is more
particularly on the message size from application side.
Let's say the application is running with 128 ranks.
Each rank is doing send() msg to rest of 127 ranks where the msg length sent is
under question.
Now after all the sends are completed, each rank will recv() msg from rest of
127 ranks.
Unless the msg length in the sending part is within eager_limit (4K size), this
program will hang.
So, based on the above scenario, my questions are:-
1. Can each of the rank send message upto 4K size successfully, i.e all 128
ranks sending (128 * 4K) bytes simultaneously?
2. If application has bigger msg to be sent by each rank, then how to derive
the send message size? Is it equal to eager_limit and each rank needs to send
multiple chunks of this size?
With Regards,
S. Biplab Raut
From: George Bosilca <bosi...@icl.utk.edu>
Sent: Tuesday, March 24, 2020 9:01 PM
To: Open MPI Users <users@lists.open-mpi.org>
Cc: Raut, S Biplab <biplab.r...@amd.com>
Subject: Re: [OMPI users] Regarding eager limit relationship to send message
size
[CAUTION: External Email]
Biplab,
The eager is a constant for each BTL, and it represent the data that is sent
eagerly with the matching information out of the entire message. So, if the
question is how much memory is needed to store all the eager messages then the
answer will depend on the communication pattern of your application:
- applications using only blocking messages might only have 1 pending
communications per peer, so in the worst case any process will only need at
most P * eager_size memory for local storage of the eager.
- applications using non-blocking communications, there is basically no limit.
However, the good news is that you can change this limit to adapt to the needs
of your application(s).
Hope this answers your question,
George.
On Tue, Mar 24, 2020 at 1:46 AM Raut, S Biplab via users
<users@lists.open-mpi.org<mailto:users@lists.open-mpi.org>> wrote:
Dear Experts,
I would like to derive/calculate the maximum MPI send
message size possible given the known details of btl_vader_eager_limit and
number of ranks.
Can anybody explain and confirm on this?
With Regards,
S. Biplab Raut
