On Mon, 2008-05-12 at 08:44 +0300, Or Gerlitz wrote: > Thinking about it a little, I don't see how this send-with-invalidate > support is applicable for ULPs such as SRP, iSER and RDS who use the > Mellanox FMRs. > > This is because to invalidate an rkey they call fmr_unmap and what the > mlx4 (similarly for mthca) driver does at mlx4_ib_unmap_fmr() is call > mlx4_fmr_unmap() for each fmr and then issue SYNC_TPT command. > > Even if doing send-with-inv would save the ULP the indirect call to > mlx4_fmr_unmap() (which does almost nothing by itself), if it doesn't > cause the HW/FW to issue SYNC_TPT, it can not replace the call to > ib_unmap_fmr in the side that generated this rkey. And if it does cause > SYNC_TPT, the effect of amortizing the cost of this heavy command > through un-mapping on many fmrs at once is lost, correct? The outcome of send with invalidate involves an implicit "sync_tpt" like operation although it syncs the caches to invalidate only the specific memory key (as opposed to sync tpt command which has a more global nature). But I think that the idea is not to save the overhead of sync tpt commands but to provide security. Perhaps someone from RDS can add more on that.
_______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
