[ewg] Re: [ofa-general] Re: [PATCH] Request For Comments:

[Steve Wise]

Sean Hefty wrote: The requirement is mostly driven from the receiving side. For cxgb3 it is anyway... Maybe you can help me understand the spec here. If we ignore this feature for a minute, then the side that calls rdma_connect() must instead issue the first 'send' request to the server. Can the first 'send' be a 0B rdma write or read? According to the MPI IETF RFC, the initiator must send the first FPDU.  That could be anything.  The spec leaves it up to the ULP. Why wouldn't the target of that request not have to transition to connected? I don't understand this question?  What does 'transition to connected' mean? The requirement is that the responder (the side that issues the rdma_accept in rdma-cma terms) _cannot_ send an FPDU until it first receives one from the initiator.   How that is enforces is an implementation detail.  The responder driver could hold off on the ESTABLISHED event until it receives the first FPDU.  Or it could stall SQ processing until the first FPDU is received yet still indicate that the connection is ESTABLISHED. Is the issue that there's no way for the receiving FW/driver to know that this has occurred so that it can signal that the connection has been established? I.e. a client that does this must signal the server that things are ready through some out of band means. I don't understand what you're getting at exactly.  The issue is that the server doesn't know when the client receives the MPA Start Response and has successfully transitioned the connection into RDMA mode.  IF the server sends an FPDU immediately following the MPA Start Response (which is in streaming mode), then its possible for that first FPDU to get passed up to the driver/ULP as streaming mode data.  Which breaks everything.  Soooo, the spec says the server cannot send an FPDU until it first receives one and thus _knows_ the client is in RDMA mode (by virtue of the fact that the client sent and FPDU). server sends MPA Start response with "lets do RTR and send me X" where X could be 0B write, 0B read request or 0B send. Are there any restrictions where a client may not be able to issue what the server requests? E.g. the hardware doesn't issue 0B writes. Well I guess there could be.  The concensus within the iWARP vendors at Reno was that 0B read would ok.  During the previous discussion on this list shortly after Reno, issues where raised that we should allow other types. We could make the MPA start request have more info than "I can do RTR".   It could have "Here are the RTR msgs I can send".    Does that help? Steve.

_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg