The goal that this proposal is to provide underpinning for common RDMA transport CM. Thus, the API ULP (both user space and kernel space) use socket addressing.
For ULP addressing this means 5 tuple: protocol, src IP addr, src port, dst IP addr, and dst port. Port is 16 bit entity. The proposal just provide a mechanism for exchanging this 5-tuple between two sides. Which entity is responsible to "use" the proposed protocol is an interesting one. I was assuming that this will be CM. After all the proposed protocol is CM extension protocol. But it can be another entity module between CM and ULP. Its job will be taking 5 tuple and populating private data and converting dst port to SID. Since OpenIB addr.c already deals with IP to IB address translations it is a logical candidate for it. On remote side it extracts info from private data and populates socket info for Consumer and passes Consumer a pointer to Consumer private data. Another interesting place to deal with is listening point. Since it is common RDMA API, 16 bit port should be use for it also. This means that the same module should locally convert port to IB SID before passing it to CM. CM just ensures that incoming connection request which matches listening SID. While it is possible to do wildcarding on the whole SID, I had not seen it is used selectively on individual bits of a SID or a port. While SDP does the conversion to IB SID from Ethernet port, this proposal shift the responsibility for port and IP address conversion from ULP down. Now lets look at each field proposed to be moved from protocol private data to SID. Protocol version. This mean that in the future if protocol version will be bumped up we will have to change the SID on which Consumer listens on and requests sent to. Not sure how to do that without changing ULP. Does not look like a good idea. IP version. This can be incorporated into SID. But if HCA has multiple IP addresses assigned to it the listening point need to specify its IP address(es). The current verbs and/or API will have to be changed to support it. But if socket is passed to listen on it does have all the needed info. Looks fine. Ethernet Protocol. The same as the one above. Src port. Very questionable. For that listening SID must have wild card for portion of SID where SRC port is incorporated. Since ULP is not aware or ever see it, it is possible. But this pushes the definition of SID beyond it current IBTA spec statement of "similar to TCP port number". The query of listen point should also hide the wildcarded SID in this case. DAPL APIs (uDAPL and kDAPL) does not expose local IP address for listen point. An additional API can be added to support passing local socket to listen on instead of Connection Qualifier. Since it is addition no backwards compatibility issues. The current ULPs/Apps will still use "the default API address" and the protocol assigned SID as connection qualifier. The new API ensures that locally SID conversion takes place. The use of protocol defined range of SIDs ensures that remote side knows to parse private data according to proposed protocol format. Arkady Arkady Kanevsky email: [EMAIL PROTECTED] Network Appliance Inc. phone: 781-768-5395 275 Totten Pond Rd. Fax: 781-895-1195 Waltham, MA 02451-2010 central phone: 781-768-5300 > -----Original Message----- > From: Sean Hefty [mailto:[EMAIL PROTECTED] > Sent: Friday, November 11, 2005 12:43 PM > To: Kanevsky, Arkady > Cc: Sean Hefty; [EMAIL PROTECTED]; > [EMAIL PROTECTED]; [email protected] > Subject: Re: [openib-general] RE: [dat-discussions] socket > based connectionmodel for IB proposal - round 3 > > Kanevsky, Arkady wrote: > > So what you are proposing is that Listener will specify > IETF port (2 > > bytes). > > CM will generate an IB SID to listen on. That SID will have > > wildcarding for 24 bits. > > The requestor will specify: version, IP version, SRC port > and DST port. > > Based on that CM will generate the SID to send request to. > > No, the listener or requester generate the SID, not the IB CM > - the same way SDP works today. > > > It will also encode IP addresses into Private data based on > IP version. > > > > This makes IP addresses, SIDs and private data format > interdependent > > and not orthogonal which it is now. > > It also changes the meaning of SID which currently has a meaning of > > TCP port. > > I'm not proposing this. I'm merely stating that is is a > valid option to consider. The private data format and SIDs > are not orthogonal anyway. The port number's embedded in the > SID, and the SID indicates the format of the private data. > They are interdependent by definition. > > If it's okay to put the destination port number in the SID, > why not the protocol type, or IP version? > > > It also does not allow to use the private data formating > for other SIDs. > > Private data is private. It should not be owned, set, > interpreted, modified, or touched by the CM. It's up to the > service to define and use. > > What's this proposal defines is basically a 65th bit for the > service ID. If the new 65 bit SID is: > > 1 <anything> - private data has this format > 0 <anything> - private data format is unknown > > Why do we need this 65th bit? > > > It looks like a big hack. Is it worth it for extra 4 bytes > of private > > data for Consumers? > > It's a trade off between SID space and private data. > Consumers need to decide how important those extra 4 bytes are. > > - Sean > > _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
