> -----Original Message-----
> From: Sean Hefty [mailto:[EMAIL PROTECTED]
> Sent: Thursday, August 25, 2005 2:37 PM
> To: 'James Lentini'; Yaron Haviv
> Cc: [email protected]
> Subject: RE: [openib-general] RDMA connection and address translation
API
>
> >> Any way providing src/dst IPs in the CM Private data is simple, and
we
> >> can come with IBTA extension blessing that data structure as a
general
> >> way to map IP oriented protocols over IB (a 1-2 page draft at the
most)
> >> This way it can also address Caitlin concerns regarding NFS & IETF
> >> (since now it's a transport specific issue)
> >
> >How long do you estimate it would take to standardize an IP<->GID
> >mechanism (ATS, CM embedded, ...) in the IBTA? 3 months? 6 months? A
> >year?....
> >
> >Let's assume that everyone on this list is in agreement.
>
> Does anyone in the IB world disagree with adding IP addresses in the
CM
> private
> data area? Would we want to extend this concept to SIDR as well?
>
> - Sean
I send my proposal from 2004 re-send again as text (attached)
Also addresses the ServiceID issue, this can be a baseline for
discussions
Feel free to change
Yaron
Mapping of iWarp/TCP connections to InfiniBand
AUTHOR Yaron Haviv ([EMAIL PROTECTED])
VERSION 0.30, Mon June 28 2004
I. INTRODUCTION
InfiniBand and iWarp semantics are similar especially with the latest
Verb Extensions, the major difference is in the way connections are
established, iWarp uses TCP based connection establishment while
InfiniBand uses a CM for that.
Another related difference is that in iWarp a user can start in a
standard TCP mode and migrate to RDMA verbs in the middle of a session.
The following document provides a general mapping from iWarp/TCP
connection establishment to InfiniBand which can be used by ULPs over
InfiniBand or by any other future iWarp protocols, it imitates the SDP
connection establishment process and CM headers (does not require SDP,
just have the same data formats for CM messages).
II. Establishing a TCP/iWarp like connections over InfiniBand
In order to emulate an iWarp connection, it is required to open an
InfiniBand RC connection, associate it with IP addresses and TCP ports
In addition protocols may transfer control/login packets before
the migration to the RDMA mode; this requires exchanging receiver buffer
size and depth for initial usage (the ULPs will manage the flow control
for the duration of the connection).
The mapping uses the same data structures already defined for connection
establishment in SDP (IBTA Socket Direct Protocol) which accomplish the
same goal of mapping TCP Sockets addressing to InfiniBand, the non
relevant SDP fields were Reserved.
iWarp emulation CM Request (Hello) Private Data header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
04 | MID | Rsvd | bufs |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
08 | len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
12 | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16 | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
20 | MajVer| MinVer| IPVer | FlowC | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
24 | DesRemRcvSz |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
28 | LocalRcvSz |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
32 | Local Port | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
36 | Src IP (127-96) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
40 | Src IP ( 95-64) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
44 | Src IP ( 63-32) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
48 | Src IP ( 31-00) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
52 | Dst IP (127-96) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
56 | Dst IP ( 95-64) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
60 | Dst IP ( 63-32) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
64 | Dst IP ( 31-00) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 CM Hello private data structure
iWarp emulation CM Response (HelloReply) Private Data header
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
04 | MID | Rsvd | bufs |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
08 | len |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
12 | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
16 | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
20 | MajVer| MinVer| Reserved | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
24 | ActRcvSz |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1 CM Reply private data structure
All the fields mentioned have the same functionality as defined in SDP
The MID field can accept only the values 0 and 1, a new field called
FlowC is added that indicate that the ULP is responsible for the Flow
Control and the Verb API usage (unlike SDP).
MID Message Identifier: 8 bits
0 Hello message
1 Reply message
Other values are reserved
FlowC Flow Control and verb owner: 4 bits
0 Transport owns flow control and may embed a 16 byte headers
in RC Send (just like SDP does, and uses zero in that field)
1 ULP owns flow control and control the entire Send buffer
(e.g. iSER does not require any additional headers)
Other values are reserved
IB Service IDs used:
Any iWarp emulated connection will use a Service ID in the form of a
constant base Service ID (IBTA_IWARP_PREFIX) + Listener TCP Port
where IBTA_IWARP_PREFIX will be allocated by IBTA or IETF, and until then a
value
of IBTA_IWARP_PREFIX= "0x0000 0000 0002 0000" will be used
Optionally it is possible to use SDP's Service ID format, and distinguish
between SDP and raw iWarp emulation based on the FlowC field.
_______________________________________________
openib-general mailing list
[email protected]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
