Rimmer, Todd wrote: >> From: Caitlin Bestler [mailto:[EMAIL PROTECTED] >> >>> >>> Actually since WQEs are in memory, while in RTR the verbs driver >>> could build the WQEs, post them to the QP, just not issue the >>> doorbells. >> >> >> That's an implementation assumption. While it is true for every HCA >> and RNIC I am aware of it is still the type of assumption that a good >> interface should not make. The other assumption is that the HCA/RNIC >> does NOT notice that a QP has contents until the doorbell is rung. >> >> NOT ringing a doorbell does NOT guarantee that the send queue will >> NOT be processed. An implemetation that lacked full confidence in its >> doorbells might have a background scan to look for non-empty send >> queues as a safety mechanism. > > The assumption would not be in the interface, but rather in > the hardware specific implementation of the verbs driver. > Hence the appropriate assumption could be made for each > hardware model. As needed the alternate TX queue could be > implemented in the verbs driver. > >> >> I still see a very simple split. If the hardware knows that it has >> the QP fenced until RTS then there is no harm in letting send WQEs be >> generated in RTR, otherwise it returns an error. Meanwhile, the verbs >> consumer > is >> simply told not to post the the send queue until the Connection >> Established event. > > The challenge is that every consume must then implement a > alternate TX queue to handle this situation. It would be > much better to hide this race in the stack or the verbs > driver rather than expose it for error prone implementation > in every consumer (both user space and kernel consumers would be > vulnerable). > > Todd Rimmer
That assumes that there is any valid reason for an application to post send requests before the connection is established. While there is clearly a need to post receive work requests before the connection is established I cannot think of any reason why an application needs to pre-prime the send queue. Putting unneeded complexity in the definition of a hardware service just invites more hardware dependencies and eventual hardware specific bugs that will complicate life for application developers. "Don't post until the connection is established" is very simple for *both* the application and the verbs provider. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
