Re: [ewg] bug 1918 - openmpi broken due to rdma-cm changes

2010-02-08 Thread Tziporet Koren
On 2/7/2010 6:39 PM, Steve Wise wrote: If ofed-1.5.1 is based on 2.6.33 then it will get this patch automatically (assuming it goes upstream and makes 2.6.33). Or we can pull it in as a kernel_patches/fixes/ patch. OFED 1.5.1 is not based on 2.6.33, but on 2.6.30, so we need the patch

Re: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Tziporet Koren
On 2/8/2010 8:02 AM, Sean Hefty wrote: Since iWarp devices are not guaranteed to support loopback connections, prevent rdma_bind_addr from associating the loopback address with an iWarp device. Signed-off-by: Sean Heftysean.he...@intel.com Steve Have you tested this patch? When accepted

[PATCH V2 02/9] ib/iser: new recv buffer posting logic

2010-02-08 Thread Or Gerlitz
Currently, the recv buffer posting logic is based on the transactional nature of iser which allows for posting a buffer before sending a PDU. Change this to post only when the number of outstanding recv buffers is below a water mark and in a batched manner, thus simplifying and optimizing the data

[PATCH V2 03/9] ib/iser: remove atomic counter for posted recv buffers

2010-02-08 Thread Or Gerlitz
With both the posting and reaping of recv buffers being in the completion path, their outstanding number counter need not be atomic. Signed-off-by: Or Gerlitz ogerl...@voltaire.com --- drivers/infiniband/ulp/iser/iscsi_iser.h |2 +- drivers/infiniband/ulp/iser/iser_initiator.c |6

[PATCH V2 04/9] ib/iser: use different CQ for send completions

2010-02-08 Thread Or Gerlitz
Use a different CQ for send completions, where send completions are being polled by the interrupt driven recv completion handler. As such, interrupts aren't used for the send CQ. Signed-off-by: Or Gerlitz ogerl...@voltaire.com --- drivers/infiniband/ulp/iser/iscsi_iser.h |3

[PATCH V2 05/9] ib/iser: simplify send flow/descriptors

2010-02-08 Thread Or Gerlitz
Simplify and shrink the logic/code used for the send descriptors. Changes include removal of struct iser_dto which is unnecessary abstraction, use struct iser_regd_buf only for handling SCSI commands, use dma_sync instead of dma_map/unmap, etc. Signed-off-by: Or Gerlitz ogerl...@voltaire.com ---

[PATCH V2 06/9] ib/iser: use atomic allocations

2010-02-08 Thread Or Gerlitz
Two minor flows in iser's data path still use allocations, move them to be atomic as a preperation step towards moving to use libiscsi passthrough mode. Signed-off-by: Or Gerlitz ogerl...@voltaire.com --- drivers/infiniband/ulp/iser/iser_initiator.c |2 +-

[PATCH V2 09/9] remove redundant locking from iser scsi command response flow

2010-02-08 Thread Or Gerlitz
currently iser recv completion flow takes the session lock twice. optimize it to avoid the first one by letting iser_task_rdma_finalize() be called only from the cleanup_task callback invoked by iscsi_free_task, thus reducing the contention on the session lock between the scsi command submission

[PATCH RESEND V2 09/9] ib/iser: remove redundant locking from iser scsi command response flow

2010-02-08 Thread Or Gerlitz
currently iser recv completion flow takes the session lock twice. optimize it to avoid the first one by letting iser_task_rdma_finalize() be called only from the cleanup_task callback invoked by iscsi_free_task, thus reducing the contention on the session lock between the scsi command submission

Re: [PATCH v2] opensm: bug in trap report for MC create(66) and delete(67) traps

2010-02-08 Thread Hal Rosenstock
On Sun, Feb 7, 2010 at 4:47 AM, Eli Dorfman (Voltaire) dorfman@gmail.com wrote: Hal Rosenstock wrote: On Fri, Feb 5, 2010 at 9:18 AM, Eli Dorfman dorfman@gmail.com wrote: On Thu, Feb 4, 2010 at 10:52 PM, Hal Rosenstock hal.rosenst...@gmail.com wrote: On Thu, Feb 4, 2010 at 12:43 PM,

[PATCH] Add new device IDs for ConnectX VPI HCAs

2010-02-08 Thread Vladimir Sokolovsky
Signed-off-by: Vladimir Sokolovsky v...@mellanox.co.il --- src/mlx4.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/src/mlx4.c b/src/mlx4.c index 1295c53..973df68 100644 --- a/src/mlx4.c +++ b/src/mlx4.c @@ -66,6 +66,8 @@ struct { HCA(MELLANOX, 0x6354), /*

Re: [ewg] bug 1918 - openmpi broken due to rdma-cm changes

2010-02-08 Thread Steve Wise
Tziporet Koren wrote: On 2/7/2010 6:39 PM, Steve Wise wrote: If ofed-1.5.1 is based on 2.6.33 then it will get this patch automatically (assuming it goes upstream and makes 2.6.33). Or we can pull it in as a kernel_patches/fixes/ patch. OFED 1.5.1 is not based on 2.6.33, but on 2.6.30, so

Re: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Steve Wise
This patch doesn't solve the openmpi/IB regression. So for OFED, IMO, we need a different patch... Tziporet Koren wrote: On 2/8/2010 8:02 AM, Sean Hefty wrote: Since iWarp devices are not guaranteed to support loopback connections, prevent rdma_bind_addr from associating the loopback

Re: [PATCH v2] opensm: bug in trap report for MC create(66) and delete(67) traps

2010-02-08 Thread Eli Dorfman (Voltaire)
Hal Rosenstock wrote: On Sun, Feb 7, 2010 at 4:47 AM, Eli Dorfman (Voltaire) dorfman@gmail.com wrote: Hal Rosenstock wrote: On Fri, Feb 5, 2010 at 9:18 AM, Eli Dorfman dorfman@gmail.com wrote: On Thu, Feb 4, 2010 at 10:52 PM, Hal Rosenstock hal.rosenst...@gmail.com wrote: On Thu,

Re: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Roland Dreier
This patch doesn't solve the openmpi/IB regression. So for OFED, IMO, we need a different patch... If this doesn't solve the regression the we should have a different patch for upstream too. The goal for 2.6.33 should be to keep open mpi working, even if that requires us to go back to old

Re: [RFC Patch] net: reserve ports for applications using fixed port numbers

2010-02-08 Thread Octavian Purdila
On Monday 08 February 2010 05:21:50 you wrote: Octavian Purdila wrote: On Friday 05 February 2010 06:45:38 you wrote: Again, using bitmap algorithm is not a problem and it's better, the problem is sysctl interface, how would you plan to interact with users via sysctl/proc if you use

Re: Patch series from Dec 2009 which needs reviewing/applying

2010-02-08 Thread Roland Dreier
[PATCH 0/2] fix SRQ WQE buffer initialization in liblmx4 and in mlx4_ib http://www.spinics.net/lists/linux-rdma/msg01911.html This isn't a patch AFAICT. [PATCH 1/2] libmlx4: initialize SRQ scatter entries when creating an SRQ http://www.spinics.net/lists/linux-rdma/msg01912.html Just

Re: [PATCH v2] opensm: bug in trap report for MC create(66) and delete(67) traps

2010-02-08 Thread Hal Rosenstock
On Mon, Feb 8, 2010 at 11:05 AM, Eli Dorfman (Voltaire) dorfman@gmail.com wrote: Hal Rosenstock wrote: On Sun, Feb 7, 2010 at 4:47 AM, Eli Dorfman (Voltaire) dorfman@gmail.com wrote: Hal Rosenstock wrote: On Fri, Feb 5, 2010 at 9:18 AM, Eli Dorfman dorfman@gmail.com wrote: On

Re: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Steve Wise
Jason Gunthorpe wrote: On Mon, Feb 08, 2010 at 08:52:10AM -0800, Roland Dreier wrote: This patch doesn't solve the openmpi/IB regression. So for OFED, IMO, we need a different patch... If this doesn't solve the regression the we should have a different patch for upstream too. The

[PATCH RFC] Here is a not tested patch that I think removes support for binding

2010-02-08 Thread Steve Wise
to 127.0.0.1. Sean will this work? If we agree to do this for 2.6.33, then I'll build/test this and resubmit. rdma/cma: Disallow binding rdma endpoints to 127.0.0.1. Currently this functionality breaks openmpi. Once openmpi is fixed to correctly ignore 127.0.0.1 as a valid external rdma

RE: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Sean Hefty
IMO 127.0.0.1 should be for SW loopback, not HW RDMA loopback. I disagree, but what does it matter? So, we add a 'software' loopback that uses 127.0.0.1. Openmpi still wouldn't work. I will commit to get the fix in openmpi asap. If we don't care if the fix is in the kernel or user space, then

Re: [ewg] [PATCH] [for-2.6.33] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Steve Wise
Sean Hefty wrote: IMO 127.0.0.1 should be for SW loopback, not HW RDMA loopback. I disagree, but what does it matter? So, we add a 'software' loopback that uses 127.0.0.1. Openmpi still wouldn't work. I guess that's true. I will commit to get the fix in openmpi asap. If

Re: [PATCH v2 00/18] Increase maximum number of Infiniband HCAs per system

2010-02-08 Thread Alex Chiang
Hi Roland, Any thoughts on this patch series? Thanks, /ac * Alex Chiang achi...@hp.com: This is v2 of a patch series that increases the maximum number of IB HCAs supported per system. The original mail thread is here: http://lkml.org/lkml/2010/1/29/346 One note, I decided to

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Jeff Squyres
Sorry -- I missed many of these mails today due to mail filtering (don't ask). FWIW: - I'm not opposed to adding LOOPBACK checks into OMPI to avoid this problem (I'm waiting for a patch, actually). I'm just saying that we're not going to get a release out immediately with this fix. Our next

[PATCH resend] rdma/cma: Disallow binding rdma endpoints to 127.0.0.1.

2010-02-08 Thread Steve Wise
Here is a not tested patch that I think removes support for binding to 127.0.0.1. Sean will this work? If we agree to do this for 2.6.33, then I'll build/test this and resubmit. rdma/cma: Disallow binding rdma endpoints to 127.0.0.1. Currently this functionality breaks openmpi. Once

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Jeff Squyres
On Feb 8, 2010, at 5:09 PM, Jason Gunthorpe wrote: DESCRIPTION Associates a source address with an rdma_cm_id. The address may be wildcarded. If binding to a specific local address, the rdma_cm_id will also be bound to a local RDMA device. This statement is trying

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Jeff Squyres
On Feb 8, 2010, at 5:13 PM, Sean Hefty wrote: Are you certain that rdma_bind_addr does NOT work with 127.0.0.1, and that this is now the problem? It does appear to work on OFED 1.4 and on 2.6.26 based on ucmatose. Is the problem really with rdma_bind_addr succeeding, or with

RE: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Sean Hefty
On my OFED 1.4.1 RHEL4u6 systems, rdma_bind_addr() fails when attempting to bind to 127.0.0.1 per the email I sent Friday: http://www.spinics.net/lists/linux-rdma/msg02568.html This is what I see over IB on 2.6.26, with a couple extra prints added to cmatose:

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Steve Wise
Sean, can you try openmpi? It fails for me, and yet ucmatose succeeds. I don't understand the difference yet... Sean Hefty wrote: On my OFED 1.4.1 RHEL4u6 systems, rdma_bind_addr() fails when attempting to bind to 127.0.0.1 per the email I sent Friday:

RE: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Sean Hefty
Sean, can you try openmpi? It fails for me, and yet ucmatose succeeds. I don't understand the difference yet... I believe the issue is that rdma_bind_addr succeeds (returns 0), but no device is assigned to the rdma_cm_id (verbs field is NULL). This was a change from commit

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Jeff Squyres
On Feb 8, 2010, at 6:48 PM, Sean Hefty wrote: rc = rdma_bind_addr(cm_id, ipaddr); if (rc || !cm_id-verbs) { rc = OMPI_SUCCESS; goto out3; } Ah, yes! Per the OMPI code you cited, I amended my printf's and see: [svbu-mpi.cisco.com:19315] FAILED to bind to 127.0.0.1:

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Pradeep Satyanarayana
Jeff Squyres wrote: On Feb 8, 2010, at 7:30 PM, Pradeep Satyanarayana wrote: elm3b199:/usr/lib # /usr/mpi/gcc/openmpi-1.4.1/bin/mpirun -np 2 --bynode --mca btl_openib_cpc_include rdmacm ring -- mpirun was unable to

Re: [ewg] rdma/cm: disallow loopback address for iwarp devices

2010-02-08 Thread Jeff Squyres
On Feb 8, 2010, at 7:50 PM, Pradeep Satyanarayana wrote: No, there is none. I got this command from one of the mails in the thread. What should I use instead? You need to compile and run an MPI program. ring is a typical test program that sends a message around in a ring. I think that OFED

Re: [PATCH] opensm: Add a name to IB subnet and include it in syslog messages

2010-02-08 Thread Arputham Benjamin
Hi Sasha, I have incorporated your feedback and sent you the modified patch with a new subject line [PATCH V2] opensm: Add option to specify prefix to syslog messages Regards, Benjamin Sasha Khapyorsky wrote: Hi Benjamin, On 14:16 Thu 21 Jan , Arputham Benjamin wrote: Added a text