RE: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-13 Thread Sean Hefty
I wasn't sure if you actually run the whole test or just let rdma_bind to be called and see the above. Anyway, if you send me a patch with the prints you've added, I can repeat it in my setup and we'll see. I let ucmatose run successfully. It's kind of a hassel for me to generate a patch for

RE: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-12 Thread Sean Hefty
It seems that even when the rdma-cm consumer binds to a specific address, the rdma-cm address resolution code follows the order of the devices/rules in routing table. So the user can't really dictate an outgoing interface based on the src address provided to

Re: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-11 Thread Or Gerlitz
On Thu, Feb 5, 2009 at 1:44 PM, Or Gerlitz ogerl...@voltaire.com wrote: It seems that even when the rdma-cm consumer binds to a specific address, the rdma-cm address resolution code follows the order of the devices/rules in routing table. So the user can't really dictate an outgoing interface

RE: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-11 Thread Sean Hefty
Did you had the chance to look into that? Not yet - but should be able to look into it by the end of the week. From what Jason said, it sounds like ip_dev_find() doesn't behave like I was expecting. - Sean ___ general mailing list

Re: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-11 Thread Or Gerlitz
Sean Hefty wrote: Not yet - but should be able to look into it by the end of the week. From what Jason said, it sounds like ip_dev_find() doesn't behave like I was expecting. OK, thanks for the update. Or. ___ general mailing list

Re: [ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-06 Thread Or Gerlitz
ucmatose allows binding to a specific address using -b. I haven't used rds-ping to know if it's the same as -I in that case. I don't have any systems myself with dual HCAs; I don't think they have enough slots to support more than one. Hi Sean, ucmatose doesn't do anything with the

Re: [ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-06 Thread Richard Frank
I played around with this a bit more yesterday - and it looks like rdma_bind_addr()-rdma_resolve_ip()-ip_dev_find() is always returning the first matching entry in the routing table... even though we are providing the source ip for the bind... Keeping in mind that both IB ports have IPs on

RE: [ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-06 Thread Sean Hefty
ucmatose doesn't do anything with the address provided with the -b param on its --active-- side, where this problem takes place. It passes the address into rdma_resolve_addr() as the source address, which results in binding to that address. - Sean ___

Re: [ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-06 Thread Or Gerlitz
On Fri, Feb 6, 2009 at 8:10 PM, Sean Hefty sean.he...@intel.com wrote: It passes the address into rdma_resolve_addr() as the source address, which results in binding to that address. OK, I managed to reproduce the problem with ucmatose in the same manner it happened with rds-ping: two running

Re: [ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-06 Thread Jason Gunthorpe
On Fri, Feb 06, 2009 at 12:05:48PM -0500, Richard Frank wrote: I played around with this a bit more yesterday - and it looks like rdma_bind_addr()-rdma_resolve_ip()-ip_dev_find() is always returning the first matching entry in the routing table... even though we are providing the source ip

Re: [ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-06 Thread Richard Frank
Interesting - Andy Grover pointed this out too - and I totally (as usual) missed the point. :( Jason Gunthorpe wrote: On Fri, Feb 06, 2009 at 12:05:48PM -0500, Richard Frank wrote: I played around with this a bit more yesterday - and it looks like

Re: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-05 Thread Or Gerlitz
Or Gerlitz wrote: Rick Frank who brought this to my attention, also handed me this patch which is claimed to workaround this issue, --- ofa_kernel-1.3.1.orig/drivers/infiniband/core/addr.c +++ ofa_kernel-1.3.1/drivers/infiniband/core/addr.c @@ -174,15 +174,29 @@ static int

[ofa-general] RE: pick the outgoing HCA based on the IP used for bind

2009-02-05 Thread Sean Hefty
Rick Frank who brought this to my attention, also handed me this patch which is claimed to workaround this issue, its badly formatted and I couldn't really understand what it does. I hoped to be able and reproduce this with rping or ucmatose, but neither allow me to specify a -I address to the

Re: [ofa-general] Re: pick the outgoing HCA based on the IP used for bind

2009-02-05 Thread Jason Gunthorpe
On Thu, Feb 05, 2009 at 02:03:42PM +0200, Or Gerlitz wrote: Or Gerlitz wrote: Rick Frank who brought this to my attention, also handed me this patch which is claimed to workaround this issue, +++ ofa_kernel-1.3.1/drivers/infiniband/core/addr.c @@ -174,15 +174,29 @@ static int