[EMAIL PROTECTED] wrote: > James and Arkady, > > > > DAPL provides a generalized abstraction to RDMA capable > transports. As a generalized abstraction, it cannot exploit > the unique properties that many of the underlying > platforms/interconnects can provide so I would like to > propose a simple (minimum impact on libdat) extensible > interface to uDAPL that will allow vendors to expose such > capabilities. I am looking for feedback, especially from the > DAT collaborative. I have included both a design document and > actual working code as a reference. > > > > The patch provides a fully tested DAT and DAPL library > (openib_cma) set with the following provider extensions: > > > > DAT_RETURN > dat_ep_post_write_immed( > IN DAT_EP_HANDLE ep_handle, > > IN DAT_COUNT num_segments > IN DAT_LMR_TRIPLET *local_iov, > IN DAT_DTO_COOKIE user_cookie, > > IN DAT_RMR_TRIPLE *remote_iov, > IN DAT_UINT32 immediate_data, > IN DAT_COMPLETION_FLAGS completion_flags); > >
There was an earlier discussion on "extended RDMA Writes" that could optionally include immediate data and/or rdma-peer-confirmed delivery. I think any discussions regarding RDMA Writes should build upon that, since we already outlined the differences between iWARP and IB and how each could be supported. To recap: it is easy to support any of the IB RDMA Write semantics over iWARP, but doing so maps to a variable number of work requests (for example, an RDMA Write *and* an RDMA Send). As I recall, there was insufficient interest to deal justify the extra complexity. But there is a base of interest in being able to do these "extended RDMA Writes" in a transport neutral fashion. > > DAT_RETURN > dat_ep_post_cmp_and_swap( > IN DAT_EP_HANDLE ep_handle, > > IN DAT_UINT64 cmp_value, > IN DAT_UINT64 swap_value, > > IN DAT_LMR_TRIPLE *local_iov, > IN DAT_DTO_COOKIE user_cookie, > > IN DAT_RMR_TRIPLE *remote_iov, > IN DAT_COMPLETION_FLAGS completion_flags); > > > > DAT_RETURN > dat_ep_post_fetch_and_add( > IN DAT_EP_HANDLE ep_handle, > > IN DAT_UINT64 add_value, > > IN DAT_LMR_TRIPLE *local_iov, > IN DAT_DTO_COOKIE user_cookie, > > IN DAT_RMR_TRIPLE *remote_iov, > IN DAT_COMPLETION_FLAGS completion_flags); > IT-API already has made a similar proposal, and the idea that atomic work requests at least exist is present in all of the verbs layer APIs (with the caveat that the response might be "illegal request"). It might be interesting to explore if there is any way that these can be defined so that they could easily be implemented at the driver or Provider layer since they are obviously not a part of an iWARP RNIC. I belive this is possible if there is no other untagged traffic (send/recv) on the same connection. Would transport neutral atomics with that restriction be of any use at the application layer? Or is just being told "no we don't support hardware-assisted atomics" a simpler attribute? Keep in mind that atomics were not omitted from iWARP by oversight. They would have been a fairly simple opcode to add to the same work queue that handles RDMA Read requests. It was the consensus of the drafters that IB atomics had not justified the silicon they required. The key question is whether the benefits of using atomics when they are hardware optimized can be retained when the overhead of checking to see if they can be used is added. _______________________________________________ openib-general mailing list [email protected] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
