Userspace apps are supposed to release all ib device resources if
they receive a fatal async event (IBV_EVENT_DEVICE_FATAL). However,
the app has no way of knowing when the device has come back up, except
to repeatedly attempt ibv_open_device() until it succeeds.
However, currently there is no protection against open succeeding when
the device is in the midst of the removal following the fatal event.
In this case, the open will succeed, but as a result the device waits
in the middle of its removal until the new app releases its ib resources
-- and the new app will not do so, since the open succeeded at a point
following the fatal event generation.
This patch adds an "active" flag to the device. The active flag is set to
false (in the interrupt context) before the "fatal" event is generated,
so any subsequent ibv_dev_open() call to the device will fail until the
device comes back up, thus preventing the above deadlock.
Signed-off-by: Jack Morgenstein <ja...@dev.mellanox.co.il>
---
Roland,
For good measure, I also set the active flag to false at mlx4_ib_remove() -- to
give some measure of protection against opening a new userspace app while the
driver
is in the process of being removed. The main issue in this patch, though, is
setting
the active flag to false before triggering the fatal event in catas.c.
This patch is for kernel 2.6.31.
-Jack
diff --git a/drivers/infiniband/hw/mlx4/main.c
b/drivers/infiniband/hw/mlx4/main.c
index ae3d759..571d231 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -342,6 +344,9 @@ static struct ib_ucontext *mlx4_ib_alloc_ucontext(struct
ib_device *ibdev,
struct mlx4_ib_alloc_ucontext_resp resp;
int err;
+ if (!dev->dev->ib_active)
+ return ERR_PTR(-EAGAIN);
+
resp.qp_tab_size = dev->dev->caps.num_qps;
resp.bf_reg_size = dev->dev->caps.bf_reg_size;
resp.bf_regs_per_page = dev->dev->caps.bf_regs_per_page;
@@ -673,6 +686,8 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
goto err_reg;
}
+ dev->ib_active = 1;
+
return ibdev;
err_reg:
@@ -698,6 +713,8 @@ static void mlx4_ib_remove(struct mlx4_dev *dev, void
*ibdev_ptr)
struct mlx4_ib_dev *ibdev = ibdev_ptr;
int p;
+ dev->ib_active = 0;
+
mlx4_ib_mad_cleanup(ibdev);
ib_unregister_device(&ibdev->ib_dev);
diff --git a/drivers/net/mlx4/catas.c b/drivers/net/mlx4/catas.c
index aa9674b..7a23beb 100644
--- a/drivers/net/mlx4/catas.c
+++ b/drivers/net/mlx4/catas.c
@@ -69,6 +69,7 @@ static void poll_catas(unsigned long dev_ptr)
if (readl(priv->catas_err.map)) {
dump_err_buf(dev);
+ dev->ib_active = 0;
mlx4_dispatch_event(dev, MLX4_DEV_EVENT_CATASTROPHIC_ERROR, 0);
if (internal_err_reset) {
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index ce7cc6c..7f6d800 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -379,6 +379,7 @@ struct mlx4_dev {
struct radix_tree_root qp_table_tree;
u32 rev_id;
char board_id[MLX4_BOARD_ID_LEN];
+ u32 ib_active;
};
struct mlx4_init_port_param {
_______________________________________________
general mailing list
general@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
