From RFC 5044, section 7.1.2 "Connection Startup Rules", Page 29:
4. MPA Responder mode implementations MUST receive and validate at
least one FPDU before sending any FPDUs or Markers.
Note: This requirement is present to allow the Initiator time to
get its receiver into Full Operation before an FPDU arrives,
avoiding potential race conditions at the Initiator. This
was also subject to some debate in the work group before
rough consensus was reached. Eliminating this requirement
would allow faster startup in some types of applications.
However, that would also make certain implementations
(particularly "dual stack") much harder.
Steve Wise wrote:
Sean Hefty wrote:
The requirement is mostly driven from the receiving side. For cxgb3 it
is anyway...
Maybe you can help me understand the spec here. If we ignore this feature for a
minute, then the side that calls rdma_connect() must instead issue the first
'send' request to the server. Can the first 'send' be a 0B rdma write or read?
According to the MPI IETF RFC, the initiator must send the first
FPDU. That could be anything. The spec leaves it up to the ULP.
Why wouldn't the target of that request not have to transition to connected?
I don't understand this question? What does 'transition to connected'
mean?
The requirement is that the responder (the side that issues the
rdma_accept in rdma-cma terms) _cannot_ send an FPDU until it first
receives one from the initiator. How that is enforces is an
implementation detail. The responder driver could hold off on the
ESTABLISHED event until it receives the first FPDU. Or it could stall
SQ processing until the first FPDU is received yet still indicate that
the connection is ESTABLISHED.
Is the issue that there's no way for the receiving FW/driver to know that this
has occurred so that it can signal that the connection has been established?
I.e. a client that does this must signal the server that things are ready
through some out of band means.
I don't understand what you're getting at exactly.
The issue is that the server doesn't know when the client receives the
MPA Start Response and has successfully transitioned the connection
into RDMA mode. IF the server sends an FPDU immediately following the
MPA Start Response (which is in streaming mode), then its possible for
that first FPDU to get passed up to the driver/ULP as streaming mode
data. Which breaks everything. Soooo, the spec says the server
cannot send an FPDU until it first receives one and thus _knows_ the
client is in RDMA mode (by virtue of the fact that the client sent and
FPDU).
server sends MPA Start response with "lets do RTR and send me X" where
X could be 0B write, 0B read request or 0B send.
Are there any restrictions where a client may not be able to issue what the
server requests? E.g. the hardware doesn't issue 0B writes.
Well I guess there could be. The concensus within the iWARP vendors
at Reno was that 0B read would ok. During the previous discussion on
this list shortly after Reno, issues where raised that we should allow
other types.
We could make the MPA start request have more info than "I can do
RTR". It could have "Here are the RTR msgs I can send". Does that
help?
Steve.
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
