First, I love Hardware Reliability.
1QP per node, this might be fine for small clusters, but what
about larger clusters, where I have an all-to-all communications
pattern ? What about say *IF* something like IB was ever designed into
a BG/L class cluster ? What about if I happen to want to run a
multi-protocol application, how is that currently handled ? *IF* I am
allowed to do that and I had an all-2-all communication pattern, looks
like I would exhaust the QP allocation table. Sounds like potentially,
as clusters/nodes increase in number, we could easily bump our heads
on some of the IBTA artificial limits set. IMHO
Instead of 1 QP for every node, don't you mean 1 QP for every
HCA/Node ? That assumes that I can have multiple HCAs per node.
Currently even at the IPoIB level, we can not do
"bonding/trunking", correct ?
Steve...
At 12:11 PM -0800 1/5/05, Diego Crupnicoff wrote:
> -----Original Message-----
> From: Woodruff, Robert J [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, January 05, 2005 3:53 PM
> To: Diego Crupnicoff; Hal Rosenstock; Peter Buckingham
> Cc: [email protected]
> Subject: RE: [openib-general] ip over ib throughtput
>
>
> The trade off is that a fully connected model requires each
> node to burn a QP for every node in the cluster and thus does
> not scale as well as the UD model. My guess is that if
> people need really high performance
> socket access, they will use SDP instead.
>
> woody
>
1 QP for every node in the cluster does not sound that bad.
SDP is a good alternative too. It has even further benefits as compared to IPoIB (built in HW reliability that eliminates the TCP/IP stack, potential for zero copy, etc). However, in terms of QP requirements, SDP would consume even more than what a connected mode IPoIB would (still not too bad given the IB HW capabilities).
Diego
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
--
Steve Poole ([EMAIL PROTECTED])
Office: 505.665.9662
Los Alamos National Laboratory
Cell: 505.699.3807
CCN - Special Projects / Advanced Development
Fax: 505.665.7793
P.O. Box 1663, MS B255
Los Alamos, NM. 87545
03149801S
Los Alamos National Laboratory
CCN - Special Projects / Advanced Development
P.O. Box 1663, MS B255
Los Alamos, NM. 87545
03149801S
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
