Jeff Squyres wrote:
On May 10, 2007, at 9:02 AM, Or Gerlitz wrote:
To start with, my hope here is at least to be able play defensive
here, that is convince you that the disadvantages are minor, where
only if this fails, would schedule myself some reading into the
ipoib-cm rfc to dig the advantages.
I ask about the advantages because OMPI currently treats QP's as
bi-directional. Having OMPI treat them at unidirectional would be a
change. I'm not against such a change, but I think we'd need to be
convinced that there are good reasons to do so. For example, on the
surface, it seems like this scheme would simply consume more QPs and
potentially more registered memory (and is therefore unattractive).
Indeed you would need two QPs per btl connection, however, for each
direction you can make the relevant QP consume ~zero resources per the
other direction, ie on side A:
for the A --> B QP : RX WR num = 0, RX SG size = 0
for the B --> A QP : TX WR num = 0, TX SG size = 0
and on side B the other way. I think that IB disallows to have zero len
WR num so you set it actually to 1. Note that since you use SRQ for
large jobs you have zero overhead for RX resources and this one TX WR
overhead for the "RX" connection on each side. This is the only memory
related overhead since you don't have to allocate any extra buffers over
what you do now.
Or.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
