On May 19, 2008, at 3:40 PM, Jon Mason wrote:
iWARP needs preposted recv buffers (or it will drop the
connection). So
this isn't a good option.
I was talking about SRQ only. You said above that iwarp does
retransmit for SRQ.
openib BTL relies on HW retransmit when using SRQ, so if iwarp
doesn't do it
reliably enough it can not be used with SRQ anyway.
How iWARP adapters behave with respect to SRQ retransmit is 100% HW
dependent.
It was my understanding that it's at least the same as how TCP handles
a dropped packet. The HW may do better than that.
The HW can queue some of the receives internally or use the HW TCP
stack to have
it retransmit. Of course, this is a BAD thing to do. The SRQ "low-
water marker"
event is the best way to handle these cases.
I disagree. I even think that the IB-retry-forever approach is bad.
Here's why:
1. Posting more at low watermark can lead to DoS-like behavior when
you have a fast sender and a slow receiver. This is exactly the
resource-exhaustion kind of behavior that a high quality MPI
implementation is supposed to avoid -- we really should to throttle
the sender somehow.
2. Resending ad infinitum simply eats up more bandwidth and takes away
network resources (e.g., switch resources) that other, legitimate
traffic. Particularly if the receiver doesn't dip into the MPI layer
for many hours. So yes, it *works*, but it's definitely sub-optimal.
--
Jeff Squyres
Cisco Systems
