[EMAIL PROTECTED] wrote: > Quoting r. Michael S. Tsirkin <[EMAIL PROTECTED]>: >> Hello! >> As was recently discussed on this list, infiniband implements an >> optional hardware end to end credits mechanism, with credits encoded >> in the AETH field in an ACK packet. >> >> The result is that an application might see performance degradation >> even if it always posts a receive work request before remote side >> posts a send work request, unless it keeps an extra receive > work request available at all times. >> Note also that most ULPs do not benefit from this mechanism because >> they all seem to implement a software based end to end flow control. >> >> A sender HCA that gets a valid credits coe in the AETH field must >> implement the e2e flow control mechanism. >> Note however, that it is always legal for the receiver side to >> disable this mechanism since the spec says: >> >>> 9.7.7.2.4 REQUESTER BEHAVIOR >>> >>> Even a responder which does generate end-to-end credits may choose >>> to send the 'invalid' code in the AETH. >> >> At least mellanox HCAs have a per-QP option to do exactly that: send >> an invalid credits code in AETH thus disabling e2e flow control for >> this half connection. >> >> I am, therefore, considering one of the following options: >> - Add a flag rq_e2e_flow_control to create qp, make e2e flow control >> disabled by default >> - Add a flag disable_rq_e2e_flow_control to create qp, make e2e flwo >> control enabled by default >> - Add a module option to mthca to enable e2e flow control for all >> QPs, disabled by default >> - Add a module option to mthca to disable e2e flow control for all >> QPs, enabled by default >> - Do nothing, assume an application always keeps an extra receive >> qork request available at all times. >> >> Ideas? > > So, how about we add a flag to ib_qp_init_attr? > Something like > responder_invalid_credits > > Which will, for RC QPs, cause the responder to always > generate invalid credits, effectively disabling hardware E2E > flow control for this receive queue. ULPs will turn this on > if they do implement flow control in software and make sure that RNR > is never generated. > > Sean?
With a properly chosen label this also documents a transport variation in a way that is helpful to application developers. Basically when this flag is set there is no hardware-layer credit-based flow control. So an iWARP device would *always* have this option set. A name like "no_hw_credit_flow_control" states the meaning to the application in a transport neutral way. An application could simply assume there is no hardware flow control, and do its own flow control instead, or it could rely on hardware flow control *if* the device attribute indicated it was enabled. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
