Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
On Thu, May 13, 2010 at 10:35:14AM -0700, Roland Dreier wrote:
> > One reason is that get_src_path_mask() may access ah_lock.
>
> I don't see that:
>
> static u8 get_src_path_mask(struct ib_device *device, u8 port_num)
> {
> struct ib_sa_device *sa_dev;
> struct ib_sa_port *port;
> unsigned long flags;
> u8 src_path_mask;
>
> sa_dev = ib_get_client_data(device, &sa_client);
> if (!sa_dev)
> return 0x7f;
>
> so if we never add the sa_client structure it just returns.
>
I see... so I have no good reason to not treat sa_query.c in the same
manner I did for multicast.c.
By the way, how do you prefer to work out all these comments. Would
you prefer a new patch set or would you prefer to push the fixed
versions to your tree?
Also, there are a few fixes that I pushed to OFED that were pushed
after I sent V8. How would you like me to communicate them?
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
> One reason is that get_src_path_mask() may access ah_lock.
I don't see that:
static u8 get_src_path_mask(struct ib_device *device, u8 port_num)
{
struct ib_sa_device *sa_dev;
struct ib_sa_port *port;
unsigned long flags;
u8 src_path_mask;
sa_dev = ib_get_client_data(device, &sa_client);
if (!sa_dev)
return 0x7f;
so if we never add the sa_client structure it just returns.
- R.
--
Roland Dreier || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
On Thu, May 13, 2010 at 08:48:06AM -0700, Roland Dreier wrote: > > Right, I'm not saying it shouldn't be in multicast.c. I'm just > wondering why you don't have the same thing for sa_query.c. > One reason is that get_src_path_mask() may access ah_lock. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
> > Also I'm wondering why you did the "count" stuff to ignore IBoE-only > > devices in multicast.c but not sa_query.c? > It seems to me the right place to put this logic as the mutlicast code > registers as an IB client and the add_one implemntation is > multicast.c. Right, I'm not saying it shouldn't be in multicast.c. I'm just wondering why you don't have the same thing for sa_query.c. - R. -- Roland Dreier || For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
On Wed, May 12, 2010 at 12:56:58PM -0700, Roland Dreier wrote:
> > @@ -1017,9 +1020,12 @@ static void ib_sa_add_one(struct ib_device *device)
> >sa_dev->end_port = e;
> >
> >for (i = 0; i <= e - s; ++i) {
> > + spin_lock_init(&sa_dev->port[i].ah_lock);
> > + if (rdma_port_link_layer(device, i + 1) !=
> IB_LINK_LAYER_INFINIBAND)
> > + continue;
>
> Not sure I understand why you move the initialization of the spinlock up
> here? It seems we ignore everything that might have to do with spinlock
> if this is an IBoE port.
We need the spinlock initialized for get_src_path_mask() which is
called by ib_init_ah_from_path() which in turn is called for IBoE
ports as well.
>
> But the larger issue is what if someone calls one of the ib_sa_XXX_query
> functions on an IBoE port? Seems we just crash on uninitialized
> structures. I guess we're assuming that the kernel is smart enough not
> to do that?
Yes, we're not calling the SA for IBoE.
>
> Also I'm wondering why you did the "count" stuff to ignore IBoE-only
> devices in multicast.c but not sa_query.c?
>
It seems to me the right place to put this logic as the mutlicast code
registers as an IB client and the add_one implemntation is
multicast.c.
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
> @@ -1017,9 +1020,12 @@ static void ib_sa_add_one(struct ib_device *device)
> sa_dev->end_port = e;
>
> for (i = 0; i <= e - s; ++i) {
> +spin_lock_init(&sa_dev->port[i].ah_lock);
> +if (rdma_port_link_layer(device, i + 1) !=
> IB_LINK_LAYER_INFINIBAND)
> +continue;
Not sure I understand why you move the initialization of the spinlock up
here? It seems we ignore everything that might have to do with spinlock
if this is an IBoE port.
But the larger issue is what if someone calls one of the ib_sa_XXX_query
functions on an IBoE port? Seems we just crash on uninitialized
structures. I guess we're assuming that the kernel is smart enough not
to do that?
Also I'm wondering why you did the "count" stuff to ignore IBoE-only
devices in multicast.c but not sa_query.c?
- R.
--
Roland Dreier || For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/index.html
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
On Wed, May 05, 2010 at 04:12:15PM -0700, Roland Dreier wrote: > > @@ -795,11 +799,12 @@ static void mcast_add_one(struct ib_device *device) > >struct mcast_device *dev; > >struct mcast_port *port; > >int i; > > + int count = 0; > > > >if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB) > >return; > > > > - dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port, > > + dev = kzalloc(sizeof *dev + device->phys_port_cnt * sizeof *port, > > > @@ -1007,7 +1010,7 @@ static void ib_sa_add_one(struct ib_device *device) > >e = device->phys_port_cnt; > >} > > > > - sa_dev = kmalloc(sizeof *sa_dev + > > + sa_dev = kzalloc(sizeof *sa_dev + > > Do you happen to remember why you needed these kmalloc -> kzalloc conversions? I can't remember why. I do have this habbit of prefering kzalloc over kmalloc because it saves troubles sometimes. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
> @@ -795,11 +799,12 @@ static void mcast_add_one(struct ib_device *device) > struct mcast_device *dev; > struct mcast_port *port; > int i; > +int count = 0; > > if (rdma_node_get_transport(device->node_type) != RDMA_TRANSPORT_IB) > return; > > -dev = kmalloc(sizeof *dev + device->phys_port_cnt * sizeof *port, > +dev = kzalloc(sizeof *dev + device->phys_port_cnt * sizeof *port, > @@ -1007,7 +1010,7 @@ static void ib_sa_add_one(struct ib_device *device) > e = device->phys_port_cnt; > } > > -sa_dev = kmalloc(sizeof *sa_dev + > +sa_dev = kzalloc(sizeof *sa_dev + Do you happen to remember why you needed these kmalloc -> kzalloc conversions? -- Roland Dreier || For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/index.html ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCHv8 02/11] ib_core: IBoE support only QP1
Since IBoE is using Ethernet as its link layer, there is no central management
entity so there is need for QP0. QP1 is still needed since it handles
communications between CM agents. This patch will create only QP1 for IBoE
ports.
Signed-off-by: Eli Cohen
---
Changes from v7:
1. Remove "always true" code
2. Fix failure to initialize port ah_lock in ib_sa_add_one
drivers/infiniband/core/agent.c | 37 +++
drivers/infiniband/core/mad.c | 27 +++---
drivers/infiniband/core/multicast.c | 25 ++---
drivers/infiniband/core/sa_query.c | 41 ++
4 files changed, 93 insertions(+), 37 deletions(-)
diff --git a/drivers/infiniband/core/agent.c b/drivers/infiniband/core/agent.c
index ae7c288..964f4fb 100644
--- a/drivers/infiniband/core/agent.c
+++ b/drivers/infiniband/core/agent.c
@@ -48,6 +48,8 @@
struct ib_agent_port_private {
struct list_head port_list;
struct ib_mad_agent *agent[2];
+ struct ib_device*device;
+ u8 port_num;
};
static DEFINE_SPINLOCK(ib_agent_port_list_lock);
@@ -58,11 +60,10 @@ __ib_get_agent_port(struct ib_device *device, int port_num)
{
struct ib_agent_port_private *entry;
- list_for_each_entry(entry, &ib_agent_port_list, port_list) {
- if (entry->agent[0]->device == device &&
- entry->agent[0]->port_num == port_num)
+ list_for_each_entry(entry, &ib_agent_port_list, port_list)
+ if (entry->device == device && entry->port_num == port_num)
return entry;
- }
+
return NULL;
}
@@ -155,14 +156,16 @@ int ib_agent_port_open(struct ib_device *device, int
port_num)
goto error1;
}
- /* Obtain send only MAD agent for SMI QP */
- port_priv->agent[0] = ib_register_mad_agent(device, port_num,
- IB_QPT_SMI, NULL, 0,
- &agent_send_handler,
- NULL, NULL);
- if (IS_ERR(port_priv->agent[0])) {
- ret = PTR_ERR(port_priv->agent[0]);
- goto error2;
+ if (rdma_port_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND)
{
+ /* Obtain send only MAD agent for SMI QP */
+ port_priv->agent[0] = ib_register_mad_agent(device, port_num,
+ IB_QPT_SMI, NULL, 0,
+ &agent_send_handler,
+ NULL, NULL);
+ if (IS_ERR(port_priv->agent[0])) {
+ ret = PTR_ERR(port_priv->agent[0]);
+ goto error2;
+ }
}
/* Obtain send only MAD agent for GSI QP */
@@ -175,6 +178,9 @@ int ib_agent_port_open(struct ib_device *device, int
port_num)
goto error3;
}
+ port_priv->device = device;
+ port_priv->port_num = port_num;
+
spin_lock_irqsave(&ib_agent_port_list_lock, flags);
list_add_tail(&port_priv->port_list, &ib_agent_port_list);
spin_unlock_irqrestore(&ib_agent_port_list_lock, flags);
@@ -182,7 +188,8 @@ int ib_agent_port_open(struct ib_device *device, int
port_num)
return 0;
error3:
- ib_unregister_mad_agent(port_priv->agent[0]);
+ if (rdma_port_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND)
+ ib_unregister_mad_agent(port_priv->agent[0]);
error2:
kfree(port_priv);
error1:
@@ -205,7 +212,9 @@ int ib_agent_port_close(struct ib_device *device, int
port_num)
spin_unlock_irqrestore(&ib_agent_port_list_lock, flags);
ib_unregister_mad_agent(port_priv->agent[1]);
- ib_unregister_mad_agent(port_priv->agent[0]);
+ if (rdma_port_link_layer(device, port_num) == IB_LINK_LAYER_INFINIBAND)
+ ib_unregister_mad_agent(port_priv->agent[0]);
+
kfree(port_priv);
return 0;
}
diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 7522008..f546ab7 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -2610,6 +2610,9 @@ static void cleanup_recv_queue(struct ib_mad_qp_info
*qp_info)
struct ib_mad_private *recv;
struct ib_mad_list_head *mad_list;
+ if (!qp_info->qp)
+ return;
+
while (!list_empty(&qp_info->recv_queue.list)) {
mad_list = list_entry(qp_info->recv_queue.list.next,
@@ -2651,6 +2654,9 @@ static int ib_mad_port_start(struct ib_mad_port_private
*port_priv)
for (i = 0; i < IB_MAD_QPS_CORE; i++) {
qp = port_priv->qp_info[i].qp;
+ if (!qp)
+ continue;
+
/*
* PKey index for QP1 is irreleva
