Dear all,
I would like you to ask for a topic that there are already many
questions but I am not familiar a lot with it. I want to understand the
behaviour of an application where there are many messages less than 64KB
(eager mode) and I use TCP network. I am trying to understand in order
to simulate this application.
For example it can be possible to have one MPI_Send of 1200 bytes after
some computation, then two messages of the same size, after computation,
etc. However according to the measurements and the profiling the cost of
the communication is less than the latency of the network. I can
understand that the cost of the MPI_Send is the copy to the buffer
however sending the message to the destination it should cost at least
the latency. So are the messages buffered in the sender and they are
sent as packet to the receiver? My tcp window is 4MB and I use the same
value for snd_buff and rcv_buff. If they are buffered in the sender what
is the criterion/algorithm? I mean if I have one message, after
computation and after again message is it possible these two messages to
be buffered from the sender point of view or this happens only on the
receiver? If there is any document/paper that I can read about this I
would be appreciate to provide me the link.
A simple example is that if I have a loop that rank 0 sends two messages
to rank 1 then the duration of the first message is bigger than the
second's one and if I increase the loop to 10 or 20 messages then all
the messages cost a lot less than the first one and also less from what
SkaMPI measures. So I am sure that it should be a buffer issue (or
something else that I can't think about).
Best regards,
Georges
