Or Gerlitz wrote:
Steve Wise wrote:
So you allocate the rkey/stag up front, allocate page_lists up front,
then as needed you populate your page list and bind it to the
rkey/stag via IB_WR_FAST_REG_MR, and invalidate that mapping via
IB_WR_INVALIDATE_MR. You can do this any number of times, and with
proper fencing, you can pipeline these mappings. Eventually when
you're done doing IO (like for NFSRDMA when the mount is unmounted)
you free up the page list(s) and mr/rkey/stag.
Yes, that was my thought as well.
Just to make sure, by "proper fencing" your understanding is that for
both IB and iWARP the ULP should not wait for the fmr work request to
complete and post the send work-request carrying the rkey/stag with the
IB_SEND_FENCE flag?
Looking in the IB spec, its seems that the fence indicator only applies
to previous rdma-read / atomic operations, eg in section 11.4.1.1 POST
SEND REQUEST it says:
Fence indicator. If the fence indicator is set, then all prior RDMA
Read and Atomic Work Requests on the queue must be completed before
starting to process this Work Request.
The fast register and invalidate work requests require that they be
completed by the device _before_ processing any subsequent work
requests. So you can post subsequent SEND WRs that utilize the rkey
without problems.
In addition, invalidate allows a local fence which means the device will
no begin processing the invalitdae until all _prior_ work requests
complete (similar to a read fence but for all prior WRS).
Steve.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
