Message: 2
Date: Tue, 23 Nov 2010 10:27:37 -0800
From: Eugene Loh <eugene....@oracle.com>
Subject: Re: [OMPI users] tool for measuring the ping with accuracy
To: Open MPI Users <us...@open-mpi.org>
Message-ID: <4cec0799.60...@oracle.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
George Markomanolis wrote:
Dear Eugene,
Thanks a lot for the answer you were right for the eager mode.
I have one more question. I am looking for an official tool to measure
the ping time, just sending a message of 1 byte or more and measure
the duration of the MPI_Send command on the rank 0 and the duration of
the MPI_Recv on rank 1. I would like to know any formal tool because I
am using also SkaMPI and the results really depend on the call of the
synchronization before the measurement starts.
So for example with synchronizing the processors, sending 1 byte, I have:
rank 0, MPI_Send: ~7 ms
rank 1, MPI_Recv: ~52 ms
where 52 ms is almost the half of the ping-pong and this is ok.
Without synchronizing I have:
rank 0, MPI_Send: ~7 ms
rank 1, MPI_Recv: ~7 ms
However I developed a simple application where the rank 0 sends 1000
messages of 1 byte to rank 1 and I have almost the second timings with
the 7 ms. If in the same application I add the MPI_Recv and MPI_Send
respectively in order to have a ping-pong application then the
ping-pong duration is 100ms (like SkaMPI). Can someone explain me why
is this happening? The ping-pong takes 100 ms and the ping without
synchronization takes 7 ms.
I'm not convinced I'm following you at all. Maybe the following helps,
though maybe it's just obvious and misses the point you're trying to make.
In a ping-pong test, you have something like this:
tsend = MPI_Wtime()
MPI_Send
tsend = MPI_Wtime() - tsend
trecv = MPI_Wtime()
MPI_Recv
trecv = MPI_Wtime() - trecv
The send time times how long it takes to get the message out of the
user's send buffer. This time is very short. In contrast, the
"receive" time mostly measures how long it takes for the ping message to
reach the peer and the pong message to return. The actual time to do
the receive processing is very short and accounts for a tiny fraction of
trecv.
If a sender sends many short messages to a receiver and the two
processes don't synchronize much, you can overlap many messages and hide
the long transit time.
Here's a simple model:
sender injects message into interconnect, MPI_Send completes (this time
is short)
message travels the interconnect to the receiver (this time is long)
receiver unpacks the message and MPI_Recv completes (this time is short)
A ping-pong test counts the long inter-process transit time. Sending
many short messages before synchronizing hides the long transit time.
Sorry if this discussion misses the point you're trying to make.
Dear Eugene,
Thanks a lot, this was what I wanted to know. Now I understood it.
Best regards,
George Markomanolis
