Document connection requests parameters in rdma_connect(), rdma_accept(), and rdma_get_cm_event(), specifically regarding initiator_depth and responder_resources.
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]> --- Doug, these are the updates to the man pages that I've made to better document some of the parameters based on your feedback. I will look at some code changes to try to trap errors setting initiator_depth and responder_resources earlier in the connection setup. I will request that these changes go into OFED 1.3. The only other request I can recall was adding the ability to migrate an id to another event channel. I still need to look into this more, but this would miss OFED 1.3. man/rdma_accept.3 | 22 +++++++++--- man/rdma_ack_cm_event.3 | 3 +- man/rdma_connect.3 | 12 +++++-- man/rdma_get_cm_event.3 | 83 ++++++++++++++++++++++++++++++++++++++++++++++- 4 files changed, 109 insertions(+), 11 deletions(-) diff --git a/man/rdma_accept.3 b/man/rdma_accept.3 index e7c3788..c0c12d8 100644 --- a/man/rdma_accept.3 +++ b/man/rdma_accept.3 @@ -11,7 +11,8 @@ rdma_accept \- Called to accept a connection request. .IP "id" 12 Connection identifier associated with the request. .IP "conn_param" 12 -Information needed to establish the connection. +Information needed to establish the connection. See CONNECTION PROPERTIES +below for details. .SH "DESCRIPTION" Called from the listening side to accept a connection or datagram service lookup request. @@ -25,13 +26,16 @@ rdma_accept is called on the new rdma_cm_id. .SH "CONNECTION PROPERTIES" The following properties are used to configure the communication and specified by the conn_param parameter when accepting a connection or datagram -communication request. Users should use the conn_param values reported in -the connection request event to determine appropriate values for these fields -when accepting. +communication request. Users should use the rdma_conn_param values reported +in the connection request event to determine appropriate values for these +fields when accepting. Users may reference the rdma_conn_param structure in +the connection event directly, or can reference their own structure. If the +rdma_conn_param structure from an event is referenced, the event must not be +acked until after this call returns. .IP private_data References a user-controlled data buffer. The contents of the buffer are -transparently passed to the remote side as part of the communication request. -May be NULL if private_data is not required. +copied and transparently passed to the remote side as part of the +communication request. May be NULL if private_data is not required. .IP private_data_len Specifies the size of the user-controlled data buffer. Note that the actual amount of data transferred to the remote side is transport dependent and may @@ -39,9 +43,15 @@ be larger than that requested. .IP responder_resources The maximum number of outstanding RDMA read and atomic operations that the local side will accept from the remote side. Applies only to RDMA_PS_TCP. +This value must be less than or equal to the local RDMA device attribute +max_qp_rd_atom and the responder_resources value reported in the connect +request event. .IP initiator_depth The maximum number of outstanding RDMA read and atomic operations that the local side will have to the remote side. Applies only to RDMA_PS_TCP. +This value must be less than or equal to the local RDMA device attribute +max_qp_init_rd_atom and the initiator_depth value reported in the connect +request event. .IP flow_control Specifies if hardware flow control should be used. Applies only to RDMA_PS_TCP. .IP retry_count diff --git a/man/rdma_ack_cm_event.3 b/man/rdma_ack_cm_event.3 index 20ccd9c..3c24357 100644 --- a/man/rdma_ack_cm_event.3 +++ b/man/rdma_ack_cm_event.3 @@ -12,6 +12,7 @@ Event to be released. .SH "DESCRIPTION" All events which are allocated by rdma_get_cm_event must be released, there should be a one-to-one correspondence between successful gets -and acks. +and acks. This call frees the event structure and any memory that it +references. .SH "SEE ALSO" rdma_get_cm_event(3), rdma_destroy_id(3) diff --git a/man/rdma_connect.3 b/man/rdma_connect.3 index a0a9095..71d5594 100644 --- a/man/rdma_connect.3 +++ b/man/rdma_connect.3 @@ -11,7 +11,7 @@ rdma_connect \- Initiate an active connection request. .IP "id" 12 RDMA identifier. .IP "conn_param" 12 -connection parameters. +connection parameters. See CONNECTION PROPERTIES below for details. .SH "DESCRIPTION" For an rdma_cm_id of type RDMA_PS_TCP, this call initiates a connection request to a remote destination. For an rdma_cm_id of type RDMA_PS_UDP, it initiates @@ -25,8 +25,8 @@ by the conn_param parameter when connecting or establishing datagram communication. .IP private_data References a user-controlled data buffer. The contents of the buffer are -transparently passed to the remote side as part of the communication request. -May be NULL if private_data is not required. +copied and transparently passed to the remote side as part of the +communication request. May be NULL if private_data is not required. .IP private_data_len Specifies the size of the user-controlled data buffer. Note that the actual amount of data transferred to the remote side is transport dependent and may @@ -34,9 +34,15 @@ be larger than that requested. .IP responder_resources The maximum number of outstanding RDMA read and atomic operations that the local side will accept from the remote side. Applies only to RDMA_PS_TCP. +This value must be less than or equal to the local RDMA device attribute +max_qp_rd_atom and remote RDMA device attribute max_qp_init_rd_atom. The +remote endpoint can adjust this value when accepting the connection. .IP initiator_depth The maximum number of outstanding RDMA read and atomic operations that the local side will have to the remote side. Applies only to RDMA_PS_TCP. +This value must be less than or equal to the local RDMA device attribute +max_qp_init_rd_atom and remote RDMA device attribute max_qp_rd_atom. The +remote endpoint can adjust this value when accepting the connection. .IP flow_control Specifies if hardware flow control should be used. Applies only to RDMA_PS_TCP. .IP retry_count diff --git a/man/rdma_get_cm_event.3 b/man/rdma_get_cm_event.3 index 252a7ab..987ead5 100644 --- a/man/rdma_get_cm_event.3 +++ b/man/rdma_get_cm_event.3 @@ -21,7 +21,88 @@ modifying the file descriptor associated with the given channel. All events that are reported must be acknowledged by calling rdma_ack_cm_event. Destruction of an rdma_cm_id will block until related events have been acknowledged. -.SH "EVENTS" +.SH "EVENT DATA" +Communication event details are returned in the rdma_cm_event structure. +This structure is allocated by the rdma_cm and released by the +rdma_ack_cm_event routine. Details of the rdma_cm_event structure are +given below. +.IP "id" 12 +The rdma_cm identifier associated with the event. If the event type is +RDMA_CM_EVENT_CONNECT_REQUEST, then this references a new id for that +communication. +.IP "listen_id" 12 +For RDMA_CM_EVENT_CONNECT_REQUEST event types, this references the +corresponding listening request identifier. +.IP "event" 12 +Specifies the type of communication event which occurred. See EVENT TYPES +below. +.IP "status" 12 +Returns any asynchronous error information associated with an event. The +status is zero unless the corresponding operation failed. +.IP "param" 12 +Provides additional details based on the type of event. Users should +select the conn or ud subfields based on the rdma_port_space of the +rdma_cm_id associated with the event. See UD EVENT DATA and CONN EVENT +DATA below. +.SH "UD EVENT DATA" +Event parameters related to unreliable datagram (UD) services: RDMA_PS_UDP and +RDMA_PS_IPOIB. The UD event data is valid for RDMA_CM_EVENT_ESTABLISHED and +RDMA_CM_EVENT_MULTICAST_JOIN events, unless stated otherwise. +.IP "private_data" 12 +References any user-specified data associated with RDMA_CM_EVENT_CONNECT_REQUEST +or RDMA_CM_EVENT_ESTABLISHED events. The data referenced by this field matches +that specified by the remote side when calling rdma_connect or rdma_accept. +This field is NULL if the event does not include private data. The buffer +referenced by this pointer is deallocated when calling rdma_ack_cm_event. +.IP "private_data_len" 12 +The size of the private data buffer. Users should note that the size of +the private data buffer may be larger than the amount of private data +sent by the remote side. Any additional space in the buffer will be +zeroed out. +.IP "ah_attr" 12 +Address information needed to send data to the remote endpoint(s). +Users should use this structure when allocating their address handle. +.IP "qp_num" 12 +QP number of the remote endpoint or multicast group. +.IP "qkey" 12 +QKey needed to send data to the remote endpoint(s). +.SH "CONN EVENT DATA" +Event parameters related to connected QP services: RDMA_PS_TCP. The +connection related event data is valid for RDMA_CM_EVENT_CONNECT_REQUEST +and RDMA_CM_EVENT_ESTABLISHED events, unless stated otherwise. +.IP "private_data" 12 +References any user-specified data associated with the event. The data +referenced by this field matches that specified by the remote side when +calling rdma_connect or rdma_accept. This field is NULL if the event +does not include private data. The buffer referenced by this pointer is +deallocated when calling rdma_ack_cm_event. +.IP "private_data_len" 12 +The size of the private data buffer. Users should note that the size of +the private data buffer may be larger than the amount of private data +sent by the remote side. Any additional space in the buffer will be +zeroed out. +.IP "responder_resources" 12 +The number of responder resources requested of the recipient. +This field matches the initiator depth specified by the remote node when +calling rdma_connect and rdma_accept. +.IP "initiator_depth" 12 +The maximum number of outstanding RDMA read/atomic operations +that the recipient may have outstanding. This field matches the responder +resources specified by the remote node when calling rdma_connect and +rdma_accept. +.IP "flow_control" 12 +Indicates if hardware level flow control is provided. +.IP "retry_count" 12 +For RDMA_CM_EVENT_CONNECT_REQUEST events only, indicates the number of times +that the recipient should retry send operations. +.IP "rnr_retry_count" 12 +The number of times that the recipient should retry receiver not ready (RNR) +NACK errors. +.IP "srq" 12 +Specifies if the sender is using a shared-receive queue. +.IP "qp_num" 12 +Indicates the remote QP number for the connection. +.SH "EVENT TYPES" The following types of communication events may be reported. .IP RDMA_CM_EVENT_ADDR_RESOLVED Address resolution (rdma_resolve_addr) completed successfully. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
