On 3/20/06, Talpey, Thomas <[EMAIL PROTECTED]> wrote: > At 06:00 PM 3/20/2006, Sean Hefty wrote: > >Can you provide more details on this statement? When are you fencing the > >send > >queue when using memory windows? > > Infiniband 101, and VI before it. Memory windows fence later operations > on the send queue until the bind completes. It's a misguided attempt to > make upper layers' job "easier" because they can post a bind and then > immediately post a send carrying the rkey. In reality, it introduces bubbles > in the send pipeline and reduces op rates dramatically. > > I argued against them in iWARP verbs, and lost. If Linux could introduce > a way to make the fencing behavior optional, I would lead the parade. > I fear most hardware is implemented otherwise.
Right - I don't think this has anything to do with the OS, Linux or otherwise. If the hardware serializes things internally, it doesn't matter what the OS wants. I suspect changing existing hardware implementations and the specs they implement is exceedingly unlikely. > Yes, I know about binding on a separate queue. That doesn't work, > because windows are semantically not fungible (for security reasons). Even if the two QPs are on the same protection domain? I thought as long as you're on the same PD, you can share MRs and MWs. You as the app are then responsible for proper synchronization to make sure you don't advertise the RKEY prematurely. >From the IB spec: "The QP, Memory Window, and Memory Region must belong to the same HCA and Protection Domain." I didn't see anything that prevented the resulting memory window from being used on a different QP within the same PD and HCA. Does iWarp behave differently? - Fab _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
