I just want to get consensus on the requirements before we get too far. One thing I forgot is that with Infiniband, the receive with immediate provides the size of the rdma write that just completed. I think we should include this in the requirements since there is ULP value here.
-arlin >-----Original Message----- >From: Kanevsky, Arkady [mailto:[EMAIL PROTECTED] >Sent: Monday, February 06, 2006 11:08 AM >To: Kanevsky, Arkady; Davis, Arlin R; Sean Hefty >Cc: [EMAIL PROTECTED]; [email protected] >Subject: RE: [dat-discussions] [openib-general] [RFC] DAT 2.0 immediatedataproposal > >Arlin, >It is too strong to state that Consumer should never send a message >equal in size to the size of immediate data. >Consumer knows from the context which one it is. >it may be based on dedicated connection, or based on ULP protocol >ordering. >Arkady > >Arkady Kanevsky email: [EMAIL PROTECTED] >Network Appliance Inc. phone: 781-768-5395 >1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 >Waltham, MA 02451 central phone: 781-768-5300 > > >> -----Original Message----- >> From: Kanevsky, Arkady >> Sent: Monday, February 06, 2006 2:05 PM >> To: Davis, Arlin R; Sean Hefty >> Cc: [EMAIL PROTECTED]; [email protected] >> Subject: RE: [dat-discussions] [openib-general] [RFC] DAT 2.0 >> immediatedataproposal >> >> Arlin, >> On Friday we agreed that receiver can not distinguish between >> 4 byte of Send or 4 bytes of Immediate data if RDMA Write >> with Immed is implemented as 2 operations: >> RDMA Write followed by Send. >> >> ULP Reciever "expects" Immediate data that is why it posts >> Recv. Depending on Transport capability it MAY complete as >> Recv or as Recv_RDMA_Write_with_Immed_in_event. >> >> Neither Provider not Consumer can distinguish between the >> cases unless there is additional info. >> >> Arkady >> >> Arkady Kanevsky email: [EMAIL PROTECTED] >> Network Appliance Inc. phone: 781-768-5395 >> 1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 >> Waltham, MA 02451 central phone: 781-768-5300 >> >> >> > -----Original Message----- >> > From: Davis, Arlin R [mailto:[EMAIL PROTECTED] >> > Sent: Monday, February 06, 2006 1:25 PM >> > To: Kanevsky, Arkady; Sean Hefty >> > Cc: [EMAIL PROTECTED]; [email protected] >> > Subject: RE: [dat-discussions] [openib-general] [RFC] DAT 2.0 >> > immediate dataproposal >> > >> > >> > Arkady, >> > >> > Your requirements are slightly different then the proposed set of >> > requirements. >> > >> > "iii) DAPL Provider does not provide any identification >> that that the >> > Receive operation matches remote RDMA Write with Immediate >> data if it >> > completes as Receive DTO. >> > >> > - It is up to an ULP to separate Receive completion of remote >> > Send from remote RDMA Write with Immediate Data." >> > >> > Tell me how this is possible? How can the application distinguish >> > between a 4 byte message and a 4 byte immediate data >> message? We would >> > have to add a new requirement... "If the provider supports >> immediate >> > data in the payload the ULP cannot send a message equal to the >> > immediate >> > data size". >> > >> > -arlin >> > >> > >-----Original Message----- >> > >From: Kanevsky, Arkady [mailto:[EMAIL PROTECTED] >> > >Sent: Monday, February 06, 2006 8:08 AM >> > >To: Sean Hefty; Davis, Arlin R >> > >Cc: [EMAIL PROTECTED]; [email protected] >> > >Subject: RE: [dat-discussions] [openib-general] [RFC] DAT >> > 2.0 immediate >> > dataproposal >> > > >> > >Here are the changes to the existing requirements chapters >> for RDMA >> > >Write with Immediate Data. >> > > >> > >Feedback please. >> > >Arkady >> > > >> > >Arkady Kanevsky email: [EMAIL PROTECTED] >> > >Network Appliance Inc. phone: 781-768-5395 >> > >1601 Trapelo Rd. - Suite 16. Fax: 781-895-1195 >> > >Waltham, MA 02451 central phone: 781-768-5300 >> > > >> > > >> > >> -----Original Message----- >> > >> From: Sean Hefty [mailto:[EMAIL PROTECTED] >> > >> Sent: Friday, February 03, 2006 7:30 PM >> > >> To: Davis, Arlin R >> > >> Cc: [EMAIL PROTECTED]; [email protected] >> > >> Subject: Re: [dat-discussions] [openib-general] [RFC] DAT 2.0 >> > >> immediate dataproposal >> > >> >> > >> Davis, Arlin R wrote: >> > >> > "Applications need an optimized mechanism to notify the >> > >> receiving end >> > >> > that RDMA write data has completed beyond the two >> > operation method >> > >> > currently used (RDMA write followed by message send). >> > This new RDMA >> > >> > write feature will support 4-bytes of inline data that >> > will be sent >> > >> >> > >> Is there any reason to restrict the size of the immediate data? >> > >> Could you define the API such that the size is variable? >> I.e. the >> > >> provider can simply give the immediate data size, with 0 >> > indicating >> > >> that it is not supported. >> > >> >> > >> > It should avoid >> > >> > any latency penalties normally associated with a two >> > >> operation method. >> > >> >> > >> I would state this as a requirement. A write followed by a send >> > >> should be pushed to the application, since they may be able to >> > >> provide additional optimizations (such as combining >> > >> operations) beyond what a provider could. >> > >> >> > >> > The initiating side must expose a 4-byte immediate data >> > >> parameter for >> > >> > the application to set the inline data. The receiving >> side must >> > >> > provide a mechanism to accept the 4-byte immediate >> data. On the >> > >> > receiving side, the write with immediate completion >> > notification is >> > >> > indicated through a receive completion. It is the >> > responsibility of >> > >> > the provider to identify to the application 4-byte >> > >> immediate data from >> > >> > a normal 4-byte send message. The inline byte ordering is >> > >> application specific." >> > >> >> > >> Requirements look good to me. >> > >> >> > >> - Sean >> > >> _______________________________________________ >> > >> openib-general mailing list >> > >> [email protected] >> > >> http://openib.org/mailman/listinfo/openib-general >> > >> >> > >> To unsubscribe, please visit >> > >> http://openib.org/mailman/listinfo/openib-general >> > >> >> > >> _______________________________________________ >> openib-general mailing list >> [email protected] >> http://openib.org/mailman/listinfo/openib-general >> >> To unsubscribe, please visit >> http://openib.org/mailman/listinfo/openib-general >> _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
