From 5f56547be86ce91dac0eed0fd6112c2277c92250 Mon Sep 17 00:00:00 2001
From: Olaf Kirch <[EMAIL PROTECTED]>
Date: Wed, 20 Feb 2008 20:43:48 +0100
Subject: [PATCH] ib_mthca: avoid recycling old FMR R_Keys too soon
When a FMR is unmapped, ib_mthca resets the map count to 0, and clears
the upper part of the R_Key which is used as the sequence counter.
This poses a problem for RDS, which uses ib_fmr_unmap as a fence
operation. RDS assumes that after issuing an unmap, the old R_Keys
will be invalid for a "reasonable" period of time. For instance, Oracle
processes uses shared memory buffers allocated from a pool of buffers.
When a process dies, we want to reclaim these buffers - but we must make sure
there are no pending RDMA operations to/from those buffers.
The only way to achieve that is by using unmap and sync the TPT.
However, when the sequence count is reset on unmap, there is a high
likelihood that a new mapping will be given the same R_Key that was
issued a few milliseconds ago.
To prevent this, we suggest to not reset the sequence count when
unmapping a FMR.
This patch addresses the issue on Arbel HCAs only - someone with
a better understanding of OFED driver internals may want to take this
patch and do it properly :-)
Signed-off-by: Olaf Kirch <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/mthca/mthca_mr.c | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/mthca/mthca_mr.c
b/drivers/infiniband/hw/mthca/mthca_mr.c
index aa6c70a..09740a1 100644
--- a/drivers/infiniband/hw/mthca/mthca_mr.c
+++ b/drivers/infiniband/hw/mthca/mthca_mr.c
@@ -835,10 +835,15 @@ void mthca_arbel_fmr_unmap(struct mthca_dev *dev, struct
mthca_fmr *fmr)
if (!fmr->maps)
return;
+ /* Do not clear the sequence count on unmap, so that we cycle
+ * through all sequence numbers before issuing the same R_Key
+ * again. */
+#if 0
key = arbel_key_to_hw_index(fmr->ibmr.lkey);
key &= dev->limits.num_mpts - 1;
key = adjust_key(dev, key);
fmr->ibmr.lkey = fmr->ibmr.rkey = arbel_hw_index_to_key(key);
+#endif
fmr->maps = 0;
--
1.5.4.rc3
--
Olaf Kirch | --- o --- Nous sommes du soleil we love when we play
[EMAIL PROTECTED] | / | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
