> -----Original Message----- > From: [EMAIL PROTECTED] [mailto:general- > [EMAIL PROTECTED] On Behalf Of Kanevsky, Arkady > Sent: Wednesday, November 28, 2007 5:30 AM > To: Steve Wise; Caitlin Bestler > Cc: Leonid Grossman; [EMAIL PROTECTED] > Subject: RE: [ofa-general] Re: iWARP peer-to-peer CM proposal > > Agree with initiator/client sending signalled 0B RDMA Read. > This will handle client side. > > Still not 100% clear on passive/server side. > Two issues which bothers me. > 1. Is "bogus" S-tag allowed for incomming RDMA ops? > I do not recall that RDDP requires that length is checked before > S-tag. > > 2. How is "verb" layer on server side knows that RDMA Read op > came and was done? Is it some back door to vendor FW? > Will this be kicked for all incoming RDMA Read ops?
As you point out, the server Verbs layer is not aware of an incoming 0B RDMA Read (or Write for that matter). Hence some kind of magic must happen in the adapter where we vendors will have a choice: a) just 'unjam' the SQ in the adapter (which means that the CM layer works as today and the server can post SQ ops before the 'unjam' is received but they won't make it to the wire) or b) send a back-door command to the CM which can then move the state machine to established only after the 'unjam' is received. Whatever is done, it cannot happen for all zero-length RDMA Read (or Write for that matter). Hence the adapter must be informed that that the next zero-length is the 'unjam' message (which also means that the server side could, in theory, omit sending the RDMA Read Response, because the RDMA Read Request was really a 'unjam'... not that I would be pushing for such an 'optimization' to avoid an extra wire message). > > Arkady Kanevsky email: [EMAIL PROTECTED] > Network Appliance Inc. phone: 781-768-5395 > 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 > Waltham, MA 02451 central phone: 781-768-5300 > > > > -----Original Message----- > > From: Steve Wise [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, November 27, 2007 7:48 PM > > To: Caitlin Bestler > > Cc: Kanevsky, Arkady; Glenn Grundstrom; Leonid Grossman; > > [EMAIL PROTECTED] > > Subject: Re: [ofa-general] Re: iWARP peer-to-peer CM proposal > > > > Caitlin Bestler wrote: > > > On Nov 27, 2007 3:58 PM, Steve Wise > > <[EMAIL PROTECTED]> wrote: > > > > > >> For the short term, I claim we just implement this as part > > of linux > > >> iwarp connection setup (mandating a 0B read be sent from > > the active > > >> side). Your proposal to add meta-data to the private data > > requires a > > >> standards change anyway and is, IMO, the 2nd phase of this whole > > >> enchilada... > > >> > > >> Steve. > > >> > > > > > > I don't see how you can have any solution here that does > > not require meta-data. > > > For non-peer-to-peer connections neither a zero length RDMA Read or > > > Write should be sent. An extraneous RDMA Read is > > particularly onerous > > > for a short lived connection that fits the classic active/passive > > > model. So *something* is telling the CMA layer that this > > connection may need an MPA unjam action. > > > If that isn't meta-data, what is it? > > > > I assumed the 0B read would _always_ be sent as part of > > establishing an iWARP connection using linux and the rdma-cm. > > > > > > > > Further, the RDMA Read solution is adequate whenever the RDMA Write > > > solution would have been (although at an unnecessary extra > > cost), but > > > as near as I can determine it is not a complete solution. If the > > > passive side needs an untagged message completion then *something* > > > needs to send it. How can the CM layer (or, I suppose, the > > ULP itself) > > > know that this untagged NOP message must be sent without meta-data? > > > > I believe at Reno we had the current rnic vendors all saying > > a SEND or 0B read will work. So: If someone has current > > iwarp HW that will _not_ > > handle this problem by doing the 0B read hack, please speak up now. > > > > > > > > As I see it, if we want to do the minimum that is required, but be > > > certain that it is adequate, we need a per-connection setup > > meta-data exchange. > > > > Are you going to prototype this? > > > > > > Steve. > > > > > > > _______________________________________________ > general mailing list > [email protected] > http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib- > general _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
