Hi Sean,
In addition to passing the hdr_len and data_len to ib_create_send_mad: rmpp_mad->rmpp_hdr.paylen_newwin = cpu_to_be32(hdr_len -
offsetof(struct ib_rmpp_mad, data) + data_len);
That's not the "real" payload length that would go into the packet; just
one segment's worth of class header. Correct ?
There are two sizes needed by the RMPP code. First, it needs the total length of the data buffer, which includes any necessary padding. This is set in the scatter-gather entry for the work request. Second, it needs to know how much of the data buffer contains valid user data.
ib_create_send_mad returns the length of the of the valid user data in a slightly encoded format. It subtracts the size of the RMPP and common MAD headers from the size (see below). The intent is that the payload length value is set to the correct value for MADs that are a single segment in length.
------------- MAD header ------------- RMPP header ------------- SA/Vendor hdr ------------- User data ------------- Pad -------------
sge length = size from MAD header to end of pad. payload = size from SA/Vendor hdr to end of user data.
If the above is correct, I'm not sure the actual payload lengths in the DATA packets are correct. I also see the padding on the last segment of a multipacket send not cleared (I integrated the part of your patch relating to the pad calculation).
I will run some tests and verify the payload values using madeye. I'm not sure why the padding isn't cleared. It may be an indication that create_send_mad isn't allocating the correct pad size.
In one test case, I see the first RMPP DATA segment sent and there is no response (ACK) from the receiver (this was due to a (receiver) test program issue). The transmitter retry depends on retries in the send_wr ud structure. Does it need to known/Is there a way to know that this failed (no ACK, etc.) when retries are exhausted or is this reliant on the receiver rerequesting or is the entire RMPP transaction treated like a UD send (e.g. unreliable) ?
The RMPP send is treated as reliable, even if no response is expected. If the send is not ACKed completely, the request will timeout, and a send failure is reported to the user. If the send completes successfully, then the user knows that it was received by the remote side; although, there's no guarantee that it was processed by anyone.
The code will retry a given segment the number of times specified by the user. If forward progress is made on the send, the retry count is reset.
Here's a summary of changes so far: 1. ib_create_send_mad either needs an additional parameter (RMPP active in current send packet) or the paylen_newwin needs to be set by the user outside of this routine.
I have this on my short term to-do list. I'm just not sure on the best approach yet...
2. Some minor ib_mad.h commentary changes for more clarity on the assumptions of the RMPP API.
_______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
