At 11:09 PM 10/3/2007, Roland Dreier wrote: >... It just keeps a list of FMRs that are available to remap, and >batches up the unregistration. It is true that an R_Key may remain >valid after an FMR is unmapped, but that's the whole point of FMRs: if >you don't batch up the real flushing to amortize the cost, they're no >better than regular MRs really.
This is an aside, but in my experience the FMR is actually a win even if it's invalidated after each use. In testing with NFS/RDMA, I believe that direct FMR manipulation via ib_map_phys_mr()/ib_unmap_fmr() was worth somewhere on the order of 35% over straight ib_reg_phys_mr()/ib_dereg_mr(). I can only assume this was because the TPT-entry setup (ib_alloc_fmr()) is avoided on a per-I/O basis. As for the pools not invalidating the R_key/handle. Speaking as a storage provider, we take data integrity darn seriously. It's my opinion that a dynamic registration scheme that doesn't include per-I/O protection is pretty much not the point of dynamic registration. In many environments however, the performance tradeoff is important - this is why I prefer an all-physical scheme to FMRs, even though it requires additional RDMA ops to handle the resulting extra scatter/gather. Additionally, FMRs don't provide byte-range protection granularity, and they're not supported by iWARP hardware (plus they're buggy as heck on early Tavors, etc). So I didn't make them a default. Tom. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
