On 10/14/2014 8:40 AM, Bart Van Assche wrote:
On 10/07/14 16:48, Sagi Grimberg wrote:
In order to support that we provide the user with an interface
to pass a scattered list of buffers to the IB core layer called
ib_indir_reg_list and provide the a new send work request opcode
called IB_WR_REG_INDIR_MR. We extend wr union with a new type of
memory registration called indir_reg where the user can place the
relevant information to perform such a memory registration.
The verbs user is expected to perform these steps:
0. Make sure that the device supports Indirect memory registration via
ib_device_cap_flag IB_DEVICE_INDIR_REGISTRATION and make sure
that ib_device_attr max_indir_reg_mr_list_len suffice for the
expected scatterlist length
1. Allocate a memory region with IB_MR_INDIRECT_REG creation flag
This is done via ib_create_mr() with mr_init_attr.flags =
IB_MR_INDIRECT_REG
2. Allocate an ib_indir_reg_list structure to hold the scattered buffers
pointers. This is done via new ib_alloc_indir_reg_list() verb
3. Populate the scattered buffers in ib_indir_reg_list.sg_list
4. Post a work request with a new opcode IB_WR_REG_INDIR_MR and
provide the populated ib_indir_reg_list
5. Perform data transfer
6. Get completion of kind IB_WC_REG_INDIR_MR (if requested)
7. Free indirect MR and ib_indir_reg_list via
ib_destroy_mr() and ib_free_indir_reg_list()
Hello Sagi,
Is there documentation available somewhere about the order in which an
HCA must execute an indirect memory registration request relative to
other work requests, similar to the "Work Request Operation Ordering"
table in the InfiniBand specification ? I think such documentation is
needed to ensure consistent behavior across HCA models.
So basically Indirect registration request generalizes fast registration
work request, so it naturally it complies to the same operation ordering
specification as fast memory registration operations.
Does it make sense to add some form of
"Documentation/infiniband/registration_ordering_rules.txt"? This should
probably include bind_mw, fastreg, indirect_reg, local_inv..
I'd like to hear more opinions here before I add it...
Roland, Sean, Steve, Chuck, Or?
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
