One more issue to discuss. Does Completion of Recv that matches RDMA Write with Immediate Data automatically sync local memory or Consumer still need to do lmr_sync_rdma_write prior to accessing RDMAed data.
Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance Inc. phone: 781-768-5395 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 Waltham, MA 02451 central phone: 781-768-5300 > -----Original Message----- > From: Caitlin Bestler [mailto:[EMAIL PROTECTED] > Sent: Tuesday, February 07, 2006 7:40 PM > To: [EMAIL PROTECTED]; Larsen, Roy K; Arlin > Davis; Hefty, Sean > Cc: [email protected] > Subject: RE: [dat-discussions] [openib-general] [RFC] > DAT2.0immediatedataproposal > > [EMAIL PROTECTED] wrote: > > We have problem no matter which option we choose. > > The current Transport Level Requirement state: > > > > There is a one-to-one correspondence between send operation on one > > Endpoint of the Connection and recv operations on the other > Endpoint > > of the Connection. > > There is no correspondence between RDMA operations on one > Endpoint of > > the Connection and recv or send data transfer operation on > the other > > Endpoint of the Connection. > > Receive operations on a Connection must be completed in the > order of > > posting of their corresponding sends. > > > > The Immediate data and Atomic ops violate these > requirements including > > ordering rules. > > > > I had started updating these rules when I generated the > first draft of > > the requirements. They are included in the enclosed pdf file. > > But they do not cover Atomic ops that also impact transport > > requirements. This chapter of the spec have not been changed since > > DAPL 1.0 and I am very concern with any changes to it. > > > > Arkady > > > > If "RDMA Write with Immediate" is viewed as being the > equivalent of doing RDMA Write and then an RDMA Send the > correspondence rule is maintained. But *only* if the "rdma > write with immediate" > has all of the semantics of a Send. > > Atomics do not violate the rules if you view them as being a > variation on an RDMA Read. They are an RDMA Read with modify. > The real question is whether it makes sense to put it in the > RDMA device. It is also not subject to emulation at a highe layer. > > With send with invalidate we know how InfiniBand *will* > support it, because of the IB 1.2 verbs. We do not know that > for atomics over iWARP. We do not know whether it will be > added, more importantly we do not know *how* it would be > added if it were added. That makes coming up with a transport > neutral definition very premature. > In particular, if atomics were added to iWARP there is a > distinct design option where it would *not* be the same work > queue as RDMA Reads (adding atomics through Queue ID 3 would > make layering on top of a current implementation much easier. > But it would mean that atomic credits would be distinct from > read credits. This is a very strong reason to defer > attempting to define RDMA Atomics in a transport neutral fashion. > > > > > > > > Yahoo! Groups Links > > <*> To visit your group on the web, go to: > http://groups.yahoo.com/group/dat-discussions/ > > <*> To unsubscribe from this group, send an email to: > [EMAIL PROTECTED] > > <*> Your use of Yahoo! Groups is subject to: > http://docs.yahoo.com/info/terms/ > > > _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
