Re: [Beowulf] building Infiniband 4x cluster questions

2011-11-13 Thread Christopher Samuel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Gilad, On 08/11/11 12:53, Gilad Shainer wrote: The latency numbers are more or less the same between the IB vendors on SDR, DDR and QDR. Mellanox is the only vendor with FDR IB for now, and with PCIe 3.0 latency are below 1us (RDMA much

Re: [Beowulf] building Infiniband 4x cluster questions

2011-11-13 Thread Gilad Shainer
There is some add latency due to the 66/64 new encoding, but overall latency is lower than QDR. MPI is below 1us. Gilad -Original Message- From: beowulf-boun...@beowulf.org [mailto:beowulf-boun...@beowulf.org] On Behalf Of Christopher Samuel Sent: Sunday, November 13, 2011 6:42 PM To:

Re: [Beowulf] building Infiniband 4x cluster questions

2011-11-13 Thread Greg Lindahl
Is that for an MPI message ? I'd heard that FDR might have higher latencies due to the coding changes that were happening - is this not the case ? When you're asking about MPI latency, are you interested in a 2-node cluster, or a big one? The usual MPI latency benchmark only uses 1 core each