> From: Tang, Changqing > Sent: Thursday, September 20, 2007 12:48 PM > To: Michael S. Tsirkin > Cc: [email protected] > Subject: [ofa-general] Can you clarify my 'RNR NAK timer' understanding ? > > > If I set RNR NAK timer to the biggest value (00000, 655.38 ms), when the > HCA recevies > a message and no receive WR outstanding, it will wait 655.38 ms, then > it sends a RNR > NAK back. > > If I post a receive WR during this waiting time, then the message will > be received, and > RNR NAK won't be sent. > > Am I right ? What is the side effect if I set RNR NAK timer a big > timer ?
No, the behavior is if an input message arrives at a QP without any receive Q entries, the receiver immediately sends an RNR NAK. The RNR NAK timeout is part of the RNR NAK packet. In response to the RNR NAK packet, the sender waits at least the given timeout before retrying the send (actual wait time could be more than the requested value). Hence supplying a large RNR NAK timeout means there will be a large penalty when you run out of receive buffers. In this case your application will "stall" for at least the RNR NAK timeout duration even if the receiver replenishes its queue shortly after the RNR NAK is sent. One alternative is a smaller RNR NAK timeout on the receiver and a large (or infinite) RNR Retry on the sender. This avoids the time delay penalty, however it wastes bandwidth in the fabric if the sender resends before the receiver has replenished its receiver queue. The best alternative, which is employed by many IB protocols such as SRP, SDP, etc is to track application level credits so the sender will not attempt to send into an empty receive Q. Obviously that requires changes to the application protocol being designed. The tradeoff of performance vs wasting bandwidth vs protocol complexity will depend on the specific application/problem you are trying to solve. Todd Rimmer _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
