Re: [OMPI users] latency #2
Georges Markomanolis wrote: Dear all, Hi again, after using MPI_Ssend seems to be what I was looking for but I would like to know more about MPI_Send. For example sending 1 byte with MPI_Send it takes 8.69 microsec but with MPI_Ssend it takes 152.9 microsec. I understand the difference but it seems that from one message's size and after their difference is not so big like trying for 518400 bytes where it needs 3515.78 microsec with MPI_Send and 3584.1 microsec with MPI_Ssend. So has is there any rule to figure out (of course it depends on the hardware) the threshold that after this size the difference between the timings of MPI_Send and MPI_Send is not so big or at least how to find it for my hardware? Most MPI implementations choose one strategy for passing short messages (sender sends, MPI implementation buffers the message, receiver receives) and long messages (sender alerts receiver, receiver replies, then sender and receiver coordinate the transfer). The first style is "eager" (sender sends eagerly without waiting for receiver to coordinate) while the second style is "rendezvous" (sender and receiver meet). The message size at which the crossover occurs can be determined or changed. In OMPI, it depends on the BTL. E.g., try "ompi_info -a | grep eager". Try the OMPI FAQ at http://www.open-mpi.org/faq/ and look at the "Tuning" categories.
Re: [OMPI users] latency #2
On 13 Sep 2010, at 12:20, Georges Markomanolis wrote: > Dear all, > > Hi again, after using MPI_Ssend seems to be what I was looking for but I > would like to know more about MPI_Send. > > For example sending 1 byte with MPI_Send it takes 8.69 microsec but with > MPI_Ssend it takes 152.9 microsec. I understand the difference but it seems > that from one message's size and after their difference is not so big like > trying for 518400 bytes where it needs 3515.78 microsec with MPI_Send and > 3584.1 microsec with MPI_Ssend. It sounds like you are measuring send overhead rather than latency, in fact as far as I know it's impossible to measure the send latency as you have no way of being able to know when to 'stop the clock', this is why ping-pong latency is always quoted. I suspect the underlying latency of the two sends is very similar to each other in practice. > So has is there any rule to figure out (of course it depends on the hardware) > the threshold that after this size the difference between the timings of > MPI_Send and MPI_Send is not so big or at least how to find it for my > hardware? Yes there is but I'm not familiar enough with OMPI to be able to tell you, I'm sure somebody can though. If my suspicion above is correct I have doubt knowing what this value is would help you at all though in terms of application performance. Ashley. -- Ashley Pittman, Bath, UK. Padb - A parallel job inspection tool for cluster computing http://padb.pittman.org.uk
[OMPI users] latency #2
Dear all, Hi again, after using MPI_Ssend seems to be what I was looking for but I would like to know more about MPI_Send. For example sending 1 byte with MPI_Send it takes 8.69 microsec but with MPI_Ssend it takes 152.9 microsec. I understand the difference but it seems that from one message's size and after their difference is not so big like trying for 518400 bytes where it needs 3515.78 microsec with MPI_Send and 3584.1 microsec with MPI_Ssend. So has is there any rule to figure out (of course it depends on the hardware) the threshold that after this size the difference between the timings of MPI_Send and MPI_Send is not so big or at least how to find it for my hardware? Thanks a lot, Best regards, Georges
