[ewg] PATCH[0/2] Add stgt to ofed1.4
Hi Vlad, These two patches add the stgt to the installation of OFED1.4 Thakns, Doron ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008)
Jeff Squyres wrote: Ditto -- I got pulled in a few different directions and I couldn't make today's call; sorry. - OMPI v1.3 - What is the proposed release schedule for OFED 1.4? Alpha - next week and Beta by end of July. - We do not yet have beta tarballs for for OMPI v1.3, but we might be able to get some in time for the OFED v1.4 schedule If you can have something in the coming two weeks it will be good. - Question about IBCM for OFED v1.4: - Any estimate on the following ticket: https://bugs.openfabrics.org/show_bug.cgi?id=963 - Any estimate on the ibcm_listen() sometimes failing issue? (I don't think there's a ticket filed against this yet) I don't think anything was done here - Sean? Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] Ofed1.3 bonding problem
Hi everybody, I have tried what is explained in the ib-bonding.txt file, but it doesn't work. What are the lines in the /etc/modprobe.conf for Redhat Enterprise linux 4 up 5? Or perhaps there is some more information needed. Could anyone help me? Regards Alicia De: Olga Shern [mailto:[EMAIL PROTECTED] Enviado el: jueves, 10 de julio de 2008 16:23 Para: Acero Fernandez Alicia CC: ewg@lists.openfabrics.org Asunto: FW: [ewg] Ofed1.3 bonding problem Hi Alicia, You are right bonding package inside OFED replaces bonding package that is installed on your OS (ib-bonding is installed under /lib/modules/uname -r/updates, therefore if you will remove it, you will have native bonding working). I assume that your OS is RH 4. Indeed in OFED 1.3 there was a bug, that Ethernet bonding didn't work on RH4, but there is a workaround in OFED 1.3.1. ib-bonding rpm includes ib-bonding.txt file that has an instruction how to configure Ethernet bonding: (/usr/share/doc/packages/ib-bonding-0.9.0/ib-bonding.txt) 3.3 Configuring Ethernet slaves --- It is not possible to have a mix of Ethernet slaves and IPoIB slaves under the same bonding master. It is possible however that a bonding master of Ethernet slaves and a bonding master of IPoIB slaves will co-exist in one machine. To configure Ethernet slaves under a bonding master use the same instructions as for IPoIB slaves (according to the OS) with one exception. When working under Redhat-AS4 do the following when configuring a bonding master with Ethernet slaves - In the master configuration file add the line SLAVEDEV=1 - In the slave configuration file leave the line TYPE=InfiniBand This bug will be fixed in OFED 1.4. Please let me know if it helps. Best Regards Olga -- Forwarded message -- From: Acero Fernandez Alicia [EMAIL PROTECTED] Date: Jul 10, 2008 10:51 AM Subject: [ewg] Ofed1.3 bonding problem To: ewg@lists.openfabrics.org Hi everybody, I am trying to install ofed1.3 in my cluster. I have done ethernet bonding in some network interfaces and I would like to install ofed1.3, but when I try it ethernet network connection is lost. I have been looking for a solution, but I have found that the module name for ethernet bonding and for infiniband bonding is the same, then perhaps it is the reason, is it true? In that case, how could I solve it? It doesn´t seem to be solved in ofed1.3.1 because I have tried to install it and the same happens. Could you help me, please? Regards Alicia Confidencialidad: Este mensaje y sus ficheros adjuntos se dirige exclusivamente a su destinatario y puede contener información privilegiada o confidencial. Si no es vd. el destinatario indicado, queda notificado de que la utilización, divulgación y/o copia sin autorización está prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente respondiendo al mensaje y proceda a su destrucción. Disclaimer: This message and its attached files is intended exclusively for its recipients and may contain confidential information. If you received this e-mail in error you are hereby notified that any dissemination, copy or disclosure of this communication is strictly prohibited and may be unlawful. In this case, please notify us by a reply and delete this email and its contents immediately. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] July 14 OFED meeting summary on OFED 1.4 features completeness status
OFED meeting summary for July 14, 2008 on 1.4 features completeness status == Summary: - Most of components are ready for the feature freeze of OFED 1.4 - We should have alpha release next week. Details: OFED 1.4 feature readiness for the freeze: - Kernel base 2.6.27 (start with 2.6.26) - 2.6.26 done, will move to 2.6.27 when rc1 available - Multi-Protocol support, Eth and IB for ConnectX - all patches submitted, to be merged this week - NFS-RDMA - kernel 2.6.26 done. Distros backports not ready yet. Will start with SLES10 and then RHEL5.1 - iSer Target - on work; merge should start this week - New iWARP verbs (fast memory thru send queue, local invalidate send work requests, Read with invalidate) - done in 2.6.27, will need backports when we move - IPoIB (LRO and RSS) - LRO - in 2.6.27, RSS - patches to be sent this week - RDS iWARP support - In progress. Still need to debug fastreg usage, but first merge has been done into Olaf's tree. - SDP (GA) - patches were sent, merged to OFED tree - Raw QP support with ConnectX - under testing - uDAPL - socket cm for scalability - done. UD extensions will be done end of week. - OpenSM - code freeze of openSM can be later since it's a closed user level apps. - ibnetdiscover library - accelerate other tools. - Congestion Control - code in ibutils done; under testing - MVAPICH 1.1 - Should have tarball by end of the month - MVAPICH2 1.2 - Should have tarball by end of the month - OpenMPI 1.3 - do not yet have beta tarballs for OMPI v1.3. Might be able to get some for the OFED v1.4 - Backports status: - ipath driver - RHEL5, SLES10 and all kernels below 2.6.18 - Qlogic - to be completed by end of the week - NFSoRDMA - we need at least RHEL5 and SLES 10 - Jeff Backer - on work - EHCA: SLES10/SP1/SP2 - IBM - done Other topics: * Woody raised the question of Dev-Con at SC08. Conclusion was that we do not see a need for a special event, but a meeting for one or two hours of all developers that will come to SC08 is a good idea. * New server setup: Jeff updated us that the new server arrived. The request is to move to the new server in 1-2 weeks. Meetings minutes on the web: http://www.openfabrics.org/txt/documentation/linux/EWG_meeting_minutes/ Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Ofed1.3 bonding problem
On 7/15/08, Acero Fernandez Alicia [EMAIL PROTECTED] wrote: Hi everybody, I have tried what is explained in the ib-bonding.txt file, but it doesn't work. What are the lines in the /etc/modprobe.conf for Redhat Enterprise linux 4 up 5? Or perhaps there is some more information needed. There is no need to add anything to /etc/modprobe.conf if your OS is RH4 UP5 Could anyone help me? Can you please send your network scripts for bonding and ifconfig output and dmesg. Thanks Olga ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] IB/ehca: Backport for RHEL 4.6
Also, fix atomic64_t backport on powerpc64.
Signed-off-by: Joachim Fenkes [EMAIL PROTECTED]
---
.../2.6.9_U6/include/asm-powerpc/atomic64.h| 61 +++
.../backport/2.6.9_U6/include/asm/atomic.h |4 +
.../2.6.9_U6/ehca_01_ibmebus_loc_code.patch| 186
3 files changed, 251 insertions(+), 0 deletions(-)
create mode 100644
kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h
create mode 100644
kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch
diff --git a/kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h
b/kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h
new file mode 100644
index 000..8036dbe
--- /dev/null
+++ b/kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h
@@ -0,0 +1,61 @@
+#ifndef __ATOMIC64_INC_BACKPORT_2_6_9__
+#define __ATOMIC64_INC_BACKPORT_2_6_9__
+
+typedef struct { volatile long counter; } atomic64_t;
+
+#define ATOMIC64_INIT(i){ (i) }
+
+#define atomic64_read(v)((v)-counter)
+#define atomic64_set(v,i) (((v)-counter) = (i))
+
+static __inline__ void atomic64_add(long a, atomic64_t *v)
+{
+ long t;
+
+ __asm__ __volatile__(
+1:ldarx %0,0,%3 # atomic64_add\n\
+ add %0,%2,%0\n\
+ stdcx. %0,0,%3 \n\
+ bne-1b
+: =r (t), =m (v-counter)
+: r (a), r (v-counter), m (v-counter)
+: cc);
+}
+
+static __inline__ long atomic64_add_return(long a, atomic64_t *v)
+{
+ long t;
+
+ __asm__ __volatile__(
+#ifdef CONFIG_SMP
+ lwsync\n
+#endif
+1:ldarx %0,0,%2 # atomic64_add_return\n\
+ add %0,%1,%0\n\
+ stdcx. %0,0,%2 \n\
+ bne-1b
+#ifdef CONFIG_SMP
+ isync\n
+#endif
+ : =r (t)
+ : r (a), r (v-counter)
+ : cc, memory);
+
+ return t;
+}
+
+static __inline__ void atomic64_inc(atomic64_t *v)
+{
+ long t;
+
+ __asm__ __volatile__(
+1:ldarx %0,0,%2 # atomic64_inc\n\
+ addic %0,%0,1\n\
+ stdcx. %0,0,%2 \n\
+ bne-1b
+: =r (t), =m (v-counter)
+: r (v-counter), m (v-counter)
+: cc);
+}
+
+#endif
diff --git a/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h
b/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h
index 11de648..2823278 100644
--- a/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h
+++ b/kernel_addons/backport/2.6.9_U6/include/asm/atomic.h
@@ -3,6 +3,10 @@
#include_next asm/atomic.h
+#ifdef __powerpc64__
+#include asm-powerpc/atomic64.h
+#endif
+
# if defined(__x86_64__)
/**
diff --git a/kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch
b/kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch
new file mode 100644
index 000..8737b79
--- /dev/null
+++ b/kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch
@@ -0,0 +1,186 @@
+diff --git a/drivers/infiniband/hw/ehca/ehca_classes.h
b/drivers/infiniband/hw/ehca/ehca_classes.h
+index 1e9e99a..fbf6e58 100644
+--- a/drivers/infiniband/hw/ehca/ehca_classes.h
b/drivers/infiniband/hw/ehca/ehca_classes.h
+@@ -112,7 +112,7 @@ struct ehca_sport {
+
+ struct ehca_shca {
+ struct ib_device ib_device;
+- struct of_device *ofdev;
++ struct ibmebus_dev *ibmebus_dev;
+ u8 num_ports;
+ int hw_level;
+ struct list_head shca_list;
+diff --git a/drivers/infiniband/hw/ehca/ehca_eq.c
b/drivers/infiniband/hw/ehca/ehca_eq.c
+index 49660df..aab80e9 100644
+--- a/drivers/infiniband/hw/ehca/ehca_eq.c
b/drivers/infiniband/hw/ehca/ehca_eq.c
+@@ -122,7 +122,7 @@ int ehca_create_eq(struct ehca_shca *shca,
+
+ /* register interrupt handlers and initialize work queues */
+ if (type == EHCA_EQ) {
+- ret = ibmebus_request_irq(eq-ist, ehca_interrupt_eq,
++ ret = ibmebus_request_irq(NULL, eq-ist, ehca_interrupt_eq,
+ IRQF_DISABLED, ehca_eq,
+ (void *)shca);
+ if (ret 0)
+@@ -130,7 +130,7 @@ int ehca_create_eq(struct ehca_shca *shca,
+
+ tasklet_init(eq-interrupt_task, ehca_tasklet_eq, (long)shca);
+ } else if (type == EHCA_NEQ) {
+- ret = ibmebus_request_irq(eq-ist, ehca_interrupt_neq,
++ ret = ibmebus_request_irq(NULL, eq-ist, ehca_interrupt_neq,
+ IRQF_DISABLED, ehca_neq,
+ (void *)shca);
+ if (ret 0)
+@@ -170,7 +170,7 @@ int ehca_destroy_eq(struct ehca_shca *shca, struct ehca_eq
*eq)
+ u64 h_ret;
+
+ spin_lock_irqsave(eq-spinlock, flags);
+- ibmebus_free_irq(eq-ist, (void *)shca);
++ ibmebus_free_irq(NULL, eq-ist, (void *)shca);
+
+ h_ret = hipz_h_destroy_eq(shca-ipz_hca_handle, eq);
+
+diff --git a/drivers/infiniband/hw/ehca/ehca_main.c
b/drivers/infiniband/hw/ehca/ehca_main.c
+index 482103e..666aafe 100644
+--- a/drivers/infiniband/hw/ehca/ehca_main.c
Re: [ewg] [PATCH 0/5] eHCA backports for OFED 1.4, part 1
Joachim Fenkes wrote: Hi, this patchset contains eHCA backports for several kernels as well as some fixes to other backports that broke on powerpc. [1/5] fixes core for 2.6.24 [2/5] fixes core for RHEL5.1 and RHEL 5.2 [3/5] adds ehca backports for 2.6.2[345] [4/5] adds ehca backports for RHEL5.1 and RHEL5.2 [5/5] adds ehca backports for SLES10 SP1 and SP2 Please review the changes, especially the core changes, and apply if you think they're okay. Thanks and regards! Joachim and Alex ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg Applied 1-5, [2/5] added for RHEL5.0 too. Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] hi from rita
Hi It`s rita again. Will you ever contact me? I made those nude pictures especially for you and I wont write to you again! If you wanna see them just drop me a line at: [EMAIL PROTECTED] ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] [PATCH] IB/ehca: Backport for RHEL 4.6
Joachim Fenkes wrote: Also, fix atomic64_t backport on powerpc64. Signed-off-by: Joachim Fenkes [EMAIL PROTECTED] --- .../2.6.9_U6/include/asm-powerpc/atomic64.h| 61 +++ .../backport/2.6.9_U6/include/asm/atomic.h |4 + .../2.6.9_U6/ehca_01_ibmebus_loc_code.patch| 186 3 files changed, 251 insertions(+), 0 deletions(-) create mode 100644 kernel_addons/backport/2.6.9_U6/include/asm-powerpc/atomic64.h create mode 100644 kernel_patches/backport/2.6.9_U6/ehca_01_ibmebus_loc_code.patch Applied, Should the same be done for 2.6.9_U4 and 2.6.9_U5? Regards. Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH 0/3] Recieve Core Affinity support
Following this email is a list of 3 patches which implements receive core affinity for IPoIB. In tests I made on my machine I got the following results with netperf TCP, 4 streams. Without RCA: 730 MB/s With RCA: 970 MB/s The patches require the patches sent by Yevegeni P. to provide support for allocation of multiple queues. I have sent an RFC a few weeks ago but it did not receive too much attention, so I implemented on the original API I sent back then. I would like this to be reviewed by the community and good comments are welcome. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] IB/core: Add support for Receive Core Affinity
Add the capability flag IB_DEVICE_IPOIB_RCA to denote devices which
support distribution of received packects to multiple receive queues.
This results in better utilization of the system CPU cores by
distributing interrupt handling between the cores. The patch adds a
new verb, ib_create_qp_range(), to create a list range of QPs with
specific alignment requirements that should be used by a consumer to
for the different receive queues.
Signed-off-by: Eli Cohen [EMAIL PROTECTED]
---
drivers/infiniband/core/verbs.c | 39 ++-
include/rdma/ib_verbs.h | 30 +-
2 files changed, 67 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/core/verbs.c b/drivers/infiniband/core/verbs.c
index a7da9be..871fb1e 100644
--- a/drivers/infiniband/core/verbs.c
+++ b/drivers/infiniband/core/verbs.c
@@ -280,6 +280,39 @@ EXPORT_SYMBOL(ib_destroy_srq);
/* Queue pairs */
+int ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr *qp_init_attr,
+ int nqps, int align, struct ib_qp *list[])
+{
+ int ret;
+ int i;
+
+ ret = pd-device-create_qp_range ?
+ pd-device-create_qp_range(pd, qp_init_attr, NULL, nqps,
+ align, list) :
+ -ENOSYS;
+
+ if (!ret) {
+ for (i = 0; i nqps; ++i) {
+ list[i]-device = pd-device;
+ list[i]-pd = pd;
+ list[i]-send_cq= qp_init_attr[i].send_cq;
+ list[i]-recv_cq= qp_init_attr[i].recv_cq;
+ list[i]-srq= qp_init_attr[i].srq;
+ list[i]-uobject= NULL;
+ list[i]-event_handler = qp_init_attr[i].event_handler;
+ list[i]-qp_context = qp_init_attr[i].qp_context;
+ list[i]-qp_type= qp_init_attr[i].qp_type;
+ atomic_inc(pd-usecnt);
+ atomic_inc(qp_init_attr[i].send_cq-usecnt);
+ atomic_inc(qp_init_attr[i].recv_cq-usecnt);
+ if (qp_init_attr[i].srq)
+ atomic_inc(qp_init_attr[i].srq-usecnt);
+ }
+ }
+ return ret;
+}
+EXPORT_SYMBOL(ib_create_qp_range);
+
struct ib_qp *ib_create_qp(struct ib_pd *pd,
struct ib_qp_init_attr *qp_init_attr)
{
@@ -331,6 +364,9 @@ static const struct {
IB_QP_QKEY),
[IB_QPT_GSI] = (IB_QP_PKEY_INDEX
|
IB_QP_QKEY),
+ },
+ .opt_param = {
+ [IB_QPT_UD] = IB_QP_RCA,
}
},
},
@@ -427,7 +463,8 @@ static const struct {
.valid = 1,
.opt_param = {
[IB_QPT_UD] = (IB_QP_CUR_STATE
|
- IB_QP_QKEY),
+ IB_QP_QKEY
|
+ IB_QP_RCA),
[IB_QPT_UC] = (IB_QP_CUR_STATE
|
IB_QP_ACCESS_FLAGS
|
IB_QP_ALT_PATH
|
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 58a0c3f..3bb391b 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -105,6 +105,7 @@ enum ib_device_cap_flags {
IB_DEVICE_UD_TSO= (119),
IB_DEVICE_MEM_MGT_EXTENSIONS= (121),
IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (122),
+ IB_DEVICE_IPOIB_RCA = (123),
};
enum ib_atomic_cap {
@@ -558,6 +559,7 @@ enum ib_qp_type {
enum ib_qp_create_flags {
IB_QP_CREATE_IPOIB_UD_LSO = 1 0,
IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK = 1 1,
+ IB_QP_CREATE_IPOIB_RCA = 1 2,
};
struct ib_qp_init_attr {
@@ -629,7 +631,8 @@ enum ib_qp_attr_mask {
IB_QP_MAX_DEST_RD_ATOMIC= (117),
IB_QP_PATH_MIG_STATE= (118),
IB_QP_CAP = (119),
- IB_QP_DEST_QPN = (120)
+ IB_QP_DEST_QPN = (120),
+ IB_QP_RCA = (121)
};
enum ib_qp_state {
@@ -648,6 +651,12 @@ enum ib_mig_state {
IB_MIG_ARMED
};
+struct rca_attr {
+ int base_qpn;
+ int num_qpn;
+ int default_qpn;
+};
+
struct ib_qp_attr {
enum ib_qp_stateqp_state;
enum ib_qp_statecur_qp_state;
@@ -674,6 +683,7 @@ struct ib_qp_attr {
u8
[ewg] [PATCH] IB/mlx4: Add support for Receive Affinity
This patch handles the implementation at the mlx4 level, of the
changes required to support RCA. It mainly handles the creation of a
range of QPs and also handles the configuration of the special RCA QP
and the required changes to the inbox parameters.
Signed-off-by: Eli Cohen [EMAIL PROTECTED]
---
drivers/infiniband/hw/mlx4/main.c|4 +-
drivers/infiniband/hw/mlx4/mlx4_ib.h |4 +
drivers/infiniband/hw/mlx4/qp.c | 228 +-
include/linux/mlx4/qp.h | 48 +++-
4 files changed, 193 insertions(+), 91 deletions(-)
diff --git a/drivers/infiniband/hw/mlx4/main.c
b/drivers/infiniband/hw/mlx4/main.c
index 0c453d0..d3c8878 100644
--- a/drivers/infiniband/hw/mlx4/main.c
+++ b/drivers/infiniband/hw/mlx4/main.c
@@ -91,7 +91,8 @@ static int mlx4_ib_query_device(struct ib_device *ibdev,
IB_DEVICE_PORT_ACTIVE_EVENT |
IB_DEVICE_SYS_IMAGE_GUID|
IB_DEVICE_RC_RNR_NAK_GEN|
- IB_DEVICE_BLOCK_MULTICAST_LOOPBACK;
+ IB_DEVICE_BLOCK_MULTICAST_LOOPBACK |
+ IB_DEVICE_IPOIB_RCA;
if (dev-dev-caps.flags MLX4_DEV_CAP_FLAG_BAD_PKEY_CNTR)
props-device_cap_flags |= IB_DEVICE_BAD_PKEY_CNTR;
if (dev-dev-caps.flags MLX4_DEV_CAP_FLAG_BAD_QKEY_CNTR)
@@ -618,6 +619,7 @@ static void *mlx4_ib_add(struct mlx4_dev *dev)
ibdev-ib_dev.query_srq = mlx4_ib_query_srq;
ibdev-ib_dev.destroy_srq = mlx4_ib_destroy_srq;
ibdev-ib_dev.post_srq_recv = mlx4_ib_post_srq_recv;
+ ibdev-ib_dev.create_qp_range = mlx4_ib_create_qp_range;
ibdev-ib_dev.create_qp = mlx4_ib_create_qp;
ibdev-ib_dev.modify_qp = mlx4_ib_modify_qp;
ibdev-ib_dev.query_qp = mlx4_ib_query_qp;
diff --git a/drivers/infiniband/hw/mlx4/mlx4_ib.h
b/drivers/infiniband/hw/mlx4/mlx4_ib.h
index ec9bf28..e26c3d6 100644
--- a/drivers/infiniband/hw/mlx4/mlx4_ib.h
+++ b/drivers/infiniband/hw/mlx4/mlx4_ib.h
@@ -103,6 +103,7 @@ struct mlx4_ib_wq {
enum mlx4_ib_qp_flags {
MLX4_IB_QP_LSO = 1 0,
MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK = 1 1,
+ MLX4_IB_QP_RCA = 1 2,
};
struct mlx4_ib_qp {
@@ -268,6 +269,9 @@ void mlx4_ib_free_srq_wqe(struct mlx4_ib_srq *srq, int
wqe_index);
int mlx4_ib_post_srq_recv(struct ib_srq *ibsrq, struct ib_recv_wr *wr,
struct ib_recv_wr **bad_wr);
+int mlx4_ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr
*qp_init_attr,
+ struct ib_udata *udata, int nqps,
+ int align, struct ib_qp *list[]);
struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd,
struct ib_qp_init_attr *init_attr,
struct ib_udata *udata);
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index fc61556..72a2d5d 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -502,9 +502,10 @@ static int create_qp_common(struct mlx4_ib_dev *dev,
struct ib_pd *pd,
if (init_attr-create_flags
IB_QP_CREATE_BLOCK_MULTICAST_LOOPBACK)
qp-flags |= MLX4_IB_QP_BLOCK_MULTICAST_LOOPBACK;
-
if (init_attr-create_flags IB_QP_CREATE_IPOIB_UD_LSO)
qp-flags |= MLX4_IB_QP_LSO;
+ if (init_attr-create_flags IB_QP_CREATE_IPOIB_RCA)
+ qp-flags |= MLX4_IB_QP_RCA;
err = set_kernel_sq_size(dev, init_attr-cap,
init_attr-qp_type, qp);
if (err)
@@ -541,11 +542,6 @@ static int create_qp_common(struct mlx4_ib_dev *dev,
struct ib_pd *pd,
}
}
- if (!sqpn)
- err = mlx4_qp_reserve_range(dev-dev, 1, 1, sqpn);
- if (err)
- goto err_wrid;
-
err = mlx4_qp_alloc(dev-dev, sqpn, qp-mqp);
if (err) {
mlx4_qp_release_range(dev-dev, sqpn, 1);
@@ -659,9 +655,6 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev,
struct mlx4_ib_qp *qp,
mlx4_qp_free(dev-dev, qp-mqp);
- if (!is_sqp(dev, qp))
- mlx4_qp_release_range(dev-dev, qp-mqp.qpn, 1);
-
mlx4_mtt_cleanup(dev-dev, qp-mtt);
if (is_user) {
@@ -678,91 +671,138 @@ static void destroy_qp_common(struct mlx4_ib_dev *dev,
struct mlx4_ib_qp *qp,
}
}
-struct ib_qp *mlx4_ib_create_qp(struct ib_pd *pd,
- struct ib_qp_init_attr *init_attr,
- struct ib_udata *udata)
+int mlx4_ib_create_qp_range(struct ib_pd *pd, struct ib_qp_init_attr
*init_attr,
+ struct ib_udata *udata, int nqps,
+ int align, struct ib_qp *list[])
{
struct mlx4_ib_dev *dev = to_mdev(pd-device);
struct
[ewg] [PATCH] IB/ipoib: Add Receive Core Affinity (RCA) support
The patch adds RCA support for datagram mode. It creates a number of
additional receive rings, equal to the number of CPU cores, and the
counts on the hardware to distribute the received packets based on
their IP address and port number; packets that are neither TCP nor
UDP, are routed to the regular UD QP. It does some arrangements in
the receive related data structs so as to unify processing of received
packets to the regular UD QP or to the any of the RCA rings. The
receive flow uses napi to provide context for simultaneous handling of
received packets.
Signed-off-by: Eli Cohen [EMAIL PROTECTED]
---
drivers/infiniband/ulp/ipoib/ipoib.h | 35 ++-
drivers/infiniband/ulp/ipoib/ipoib_ethtool.c | 63 -
drivers/infiniband/ulp/ipoib/ipoib_ib.c | 202 +++---
drivers/infiniband/ulp/ipoib/ipoib_main.c| 89 +-
drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 379 +-
5 files changed, 682 insertions(+), 86 deletions(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h
b/drivers/infiniband/ulp/ipoib/ipoib.h
index b0ffc9a..bb6534d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -105,6 +105,8 @@ enum {
MAX_SEND_CQE = 16,
IPOIB_CM_COPYBREAK= 256,
+
+ MAX_RCA_QPS = 8,
};
#defineIPOIB_OP_RECV (1ul 31)
@@ -267,6 +269,26 @@ struct ipoib_lro {
struct net_lro_desc lro_desc[IPOIB_MAX_LRO_DESCRIPTORS];
};
+struct rca_ring_ctx {
+ struct ib_qp *qp;
+ struct ib_cq *cq;
+ struct ib_wcibwc[IPOIB_NUM_WC];
+ struct napi_struct napi;
+ struct ipoib_rx_buf*rx_ring;
+ struct ib_recv_wr rx_wr;
+ struct ib_sge rx_sge[IPOIB_UD_RX_SG];
+ struct ipoib_lrolro;
+ unsigned long rx_packets;
+ unsigned long rx_dropped;
+ unsigned long rx_bytes;
+};
+
+struct ipoib_rca {
+ struct ib_qp *rca_qp;
+ int nrings;
+ struct rca_ring_ctx ring_arr[0];
+};
+
/*
* Device private locking: tx_lock protects members used in TX fast
* path (and we use LLTX so upper layers don't do extra locking).
@@ -317,7 +339,7 @@ struct ipoib_dev_priv {
unsigned int mcast_mtu;
unsigned int max_ib_mtu;
- struct ipoib_rx_buf *rx_ring;
+ struct rca_ring_ctx rx_ring;
spinlock_t tx_lock;
struct ipoib_tx_buf *tx_ring;
@@ -328,11 +350,6 @@ struct ipoib_dev_priv {
unsigned tx_outstanding;
struct ib_wc send_wc[MAX_SEND_CQE];
- struct ib_recv_wrrx_wr;
- struct ib_sgerx_sge[IPOIB_UD_RX_SG];
-
- struct ib_wc ibwc[IPOIB_NUM_WC];
-
struct list_head dead_ahs;
struct ib_event_handler event_handler;
@@ -354,7 +371,7 @@ struct ipoib_dev_priv {
struct ipoib_ethtool_st ethtool;
struct timer_list poll_timer;
- struct ipoib_lro lro;
+ struct ipoib_rca *rca;
};
struct ipoib_ah {
@@ -427,6 +444,9 @@ extern struct workqueue_struct *ipoib_workqueue;
int ipoib_poll(struct napi_struct *napi, int budget);
void ipoib_ib_completion(struct ib_cq *cq, void *dev_ptr);
void ipoib_send_comp_handler(struct ib_cq *cq, void *dev_ptr);
+void ipoib_rca_rx_handler(struct ib_cq *cq, void *ptr);
+int rca_ring_poll(struct napi_struct *napi, int budget);
+void ipoib_rca_qps_to_err(struct net_device *dev);
struct ipoib_ah *ipoib_create_ah(struct net_device *dev,
struct ib_pd *pd, struct ib_ah_attr *attr);
@@ -495,6 +515,7 @@ int ipoib_mcast_attach(struct net_device *dev, u16 mlid,
int ipoib_init_qp(struct net_device *dev);
int ipoib_transport_dev_init(struct net_device *dev, struct ib_device *ca);
void ipoib_transport_dev_cleanup(struct net_device *dev);
+int ipoib_rca_init(struct net_device *dev);
void ipoib_event(struct ib_event_handler *handler,
struct ib_event *record);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
index 66af5c1..2c64825 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ethtool.c
@@ -60,6 +60,8 @@ static int ipoib_set_coalesce(struct net_device *dev,
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
int ret;
+ int i;
+ struct rca_ring_ctx *ring;
/*
* Since IPoIB uses a single CQ for both rx and tx, we assume
@@ -78,6 +80,19 @@ static int ipoib_set_coalesce(struct net_device *dev,
return ret;
}
+ if (priv-rca) {
+ for (i = 0; i priv-rca-nrings; ++i) {
+ ring = priv-rca-ring_arr[i];
+ ret = ib_modify_cq(ring-cq,
coal-rx_max_coalesced_frames,
+
Re: [ewg] [PATCH] IB/ehca: Backport for RHEL 4.6
Vladimir Sokolovsky [EMAIL PROTECTED] wrote on 15.07.2008 17:54:19: Applied, Thanks! Should the same be done for 2.6.9_U4 and 2.6.9_U5? I'd rather not; we haven't tested those. Regards, Joachim ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
RE: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008)
- Question about IBCM for OFED v1.4: - Any estimate on the following ticket: https://bugs.openfabrics.org/show_bug.cgi?id=963 - Any estimate on the ibcm_listen() sometimes failing issue? (I don't think there's a ticket filed against this yet) I don't think anything was done here - Sean? I have been working with Jeff and the ompi developers directly on a different thread, but I don't think that ofa general is copied on it. There's a possibility that ompi is colliding with the SDP listen space, and we're exploring that option. - Sean ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] Re: [ofa-general] OFED meeting agenda for today (July 14, 2008)
On Jul 15, 2008, at 1:03 PM, Sean Hefty wrote: - Any estimate on the ibcm_listen() sometimes failing issue? (I don't think there's a ticket filed against this yet) I don't think anything was done here - Sean? I have been working with Jeff and the ompi developers directly on a different thread, but I don't think that ofa general is copied on it. There's a possibility that ompi is colliding with the SDP listen space, and we're exploring that option. Yep-- sorry for the cross posting; this one was posted before we started the thread yesterday on the OMPI list about the same issue. There's a high probability that we're doing something wrong in OMPI's IBCM support. -- Jeff Squyres Cisco Systems ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: new server for OpenFabrics
Hi all. I logged into the new server today, and upgraded it to the latest patches, and installed git and compilers. Please take a look to see if I missed anything or if you need something else. If you add anything, please use the apt package manager (or the aptitude front end) if possible. Thanks. -jeff Johann George wrote: The new server has been loaded with the base Linux O/S and is accessible on a different IP address (69.55.231.68). It has faster processors and more cores, memory and bandwidth. The plan is to configure it, move files over from the old server and once everything is working, switch IP addresses. Jeff Becker is attempting to get it configured (while also preparing NFS-RDMA for OFED 1.4). Hope to be available soon. We'll keep you updated. Johann On Tue, Jun 24, 2008 at 03:41:55PM +0300, Tziporet Koren wrote: This remind me that we suppose to have server upgrade in the comming days Johann/Jeff - any update in this? Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] backport patches for ib_ipath
The following changes allow ib_ipath to compile on everything except RHEL4. Vlad, please pull from: git.openfabrics.org/~ralphc/ofed-1.4-linux-2.6/.git ofed_kernel commit a6415e3fb0f09bbfff15d4f37cd8709a614a1482 Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:25:59 2008 -0700 BACKPORT - fix minor warnings for 2.6.16 and older kernels Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit d1560bfbd84e0a4c359438a492c77f130dd98bc9 Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:25:20 2008 -0700 BACKPORT - change signature of ipath_get_sb for 2.6.17 and earlier Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit 2b2f4ec8098928ba0ed323770c1060d06566c222 Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:24:50 2008 -0700 BACKPORT - don't clear memory a second time Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit 3c65e7f89b5d2daf1d1addc46ca9fa690fbc0ece Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:24:04 2008 -0700 BACKPORT - avoid using vmalloc_user() in 2.6.18 due to bugs. Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit 436d3d94a211f4766345ecee4925e6740b5a3eca Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:22:35 2008 -0700 BACKPORT - use old HT IRQ infrastructure on 2.6.18 and earlier Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit c0d89ab2a968137020cbf48ea393b560329974e9 Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:21:58 2008 -0700 BACKPORT - Use writev instead of aio_write in 2.6.18 and older kernels Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit b7882e7868c87160de13d808f42f909783c1a973 Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:21:19 2008 -0700 IB/ipath - revert struct vm_operations_struct nopage to fault rename This patch reverts commit 3c8450860ba9d6279dbc969633eacf99161860d9 in 2.6.25 Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit dfa49c798bcd79a9c1f7084ea1d234d68e947a4f Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:20:40 2008 -0700 BACKPORT - revert to older code for creating sysfs driver groups This reverts commit 23b9c1ab5baf368a32b7242bf110ef1f48700d04 in 2.6.25 Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit 27c62b169696746a12860c5faab712226a3613ad Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:19:50 2008 -0700 BACKPORT - revert struct class_dev to device This patch reverts commit f4e91eb4a81559da87a3843758a641b5cc590b65 in 2.6.26 Signed-off-by: Ralph Campbell [EMAIL PROTECTED] commit e8762cc6a8be9f2f2b64cd6801fe414d5838483e Author: Ralph Campbell (QLogic) [EMAIL PROTECTED] Date: Tue Jul 15 18:19:09 2008 -0700 BACKPORT - revert pid_t to struct pid pointer change This reverts commit 40d97692fbfe52ef68fa771d8121394b2210fd67 in 2.6.26. Signed-off-by: Ralph Campbell [EMAIL PROTECTED] ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] MSG ID:26373 Why wait to buy rich?
The world's largest luxury store for shoes and bags is just one click away. Recommended by thousands of satisfied customers worldwide, we carry dozens of famous brands including: ~ Louis Vuitton ~ Armani ~ Gucci ~ Prada ~ Hermes Here you will find thousands of stunning designs for shoes, and leather products, at rock bottom pricing. Prices range from just $39 to $199; quality is assured and satisfaction absolutely guaranteed. Sale ends this week, so visit us today and start pampering yourself and your loved ones! - Visit our site: www.halltip[DOT]com (copy this link and then replace [DOT] to .) ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
