, Håkon
On Mar 27, 2009, at 19:09 , Joshua mora acosta wrote:
So a way to quantify if multirail helps on network latency driven
workloads
there should be a sinthetic benchmark that can be built to show off
the impact
of balancing these requests among multiple HCAs bound to different
network
paths or core pairs like an all-to-[all,gather,scatter] or barrier
benchmark
and in theory observe half the total latency of that overall
communication.
So I think multirail will reduce latency of sinchronizations
(collective
calls) but not to latency driven point to point communications.
Well, you're wrong. As stated, we do see speedup due to increased
message rate on apps not using collectives. As to a benchmark, and all-
to-all with _many_ processes per node will show you this. ($MPI_HOME/
examples/bin/mpi_msg_rate in our distro).
I would actually claim the opposite; it will _not_ help on most
collective operations, because they perform SMP optimizations and one
process sends and receives on behalf of the other processes.
Uncorrelated messages from many processes on a single node will, on
the other hand, take advantages of the accumulated increased message
rate provided by multiple HCAs.
Håkon
Joshua
------ Original Message ------
Received: 12:37 PM CDT, 03/27/2009
From: Håkon Bugge <[email protected]>
To: Craig Tierney <[email protected]>Cc: Joshua mora acosta
<[email protected]>, [email protected], [email protected]
Subject: Re: [Beowulf] Lowered latency with multi-rail IB?
On Mar 27, 2009, at 18:20 , Craig Tierney wrote:
What about using multi-rail to increase message rate? That isn't
the same as latency, but if you put messages on both wires you
should get more.
Exactly why we saw almost 2x speedup on message rate (latency)
sensitive apps using Platform MPI. We call the technique alternating;
any communicating peer will use a HCA or port, but a single MPI
process will alternate between different HCAs (or ports), depending
on
which peer he communicates with.
Håkon
_______________________________________________
Beowulf mailing list, [email protected]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
