> From: Caitlin Bestler [mailto:[EMAIL PROTECTED] > > > > > Actually since WQEs are in memory, while in RTR the verbs driver could > > build the WQEs, post them to the QP, just not issue the doorbells. > > > That's an implementation assumption. While it is true for every HCA > and RNIC I am aware of it is still the type of assumption that a good > interface should not make. The other assumption is that the HCA/RNIC > does NOT notice that a QP has contents until the doorbell is rung. > > NOT ringing a doorbell does NOT guarantee that the send queue will NOT > be processed. An implemetation that lacked full confidence in its > doorbells > might have a background scan to look for non-empty send queues as a > safety > mechanism.
The assumption would not be in the interface, but rather in the hardware specific implementation of the verbs driver. Hence the appropriate assumption could be made for each hardware model. As needed the alternate TX queue could be implemented in the verbs driver. > > I still see a very simple split. If the hardware knows that it has the > QP fenced until RTS then there is no harm in letting send WQEs be > generated > in RTR, otherwise it returns an error. Meanwhile, the verbs consumer is > simply told not to post the the send queue until the Connection > Established > event. The challenge is that every consume must then implement a alternate TX queue to handle this situation. It would be much better to hide this race in the stack or the verbs driver rather than expose it for error prone implementation in every consumer (both user space and kernel consumers would be vulnerable). Todd Rimmer _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
