[ewg] Re: [PATCH ofed-1.3] Load iw_cxgb3 as part of ofed init.
Steve Wise wrote: Vlad, can you please review this? Is there anything else needed to get iw_cxgb3 to be loaded at init time? I tested the patched openibd and it seems to work fine. If this looks good to you, please pull this patch for ofed-1.3 from: git://www.openfabrics.org/~swise/ofed-1.3 ofed_kernel This change is long overdue... Thanks, Steve. -- Load iw_cxgb3 as part of ofed init. Signed-off-by: Steve Wise [EMAIL PROTECTED] --- Applied, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg][GIT PULL] please pull srpt ofed_1_3.git,
Vu Pham wrote: Hi Vlad, Please pull from git://git.openfabrics.org/~vu/ofed_1_3.git This pull a fix for srpt in ofed-1.3 GIT COMMIT COMMENTS srpt: avoid disconnecting/removing connection again when it is already in disconnecting state. Manipulate connection list with spinlock_irq Signed-off-by: Vu Pham [EMAIL PROTECTED] thanks, -vu ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] Report proper error code in [was: trying to reproduce the crash]
I've been struggling with crashes in mthca_arbel_map_phys for a few days
(triggered
by RDS), and I think I'm finally making some progress
mthca_fmr_alloc does this:
if (mthca_is_memfree(dev)) {
err = mthca_table_get(dev, dev-mr_table.mpt_table, key);
if (err)
goto err_out_mpt_free;
...
}
/* when we get here, err == 0 (at least for memfree cards) */
mr-mtt = __mthca_alloc_mtt(dev, list_len, dev-mr_table.fmr_mtt_buddy);
if (IS_ERR(mr-mtt))
goto err_out_table;
err_out_table:
/* clean up some */
return err;
ie we set mr-mtt to some ERR_PTR(-whatever), and return success.
The same problem exists when mailbox allocation fails.
I fixed this, using the patch below. Now I'm making some progress:
First, the kernel reports:
RDS/IB: ib_alloc_fmr failed (err=-12)
which is good - now we get a decent error code instead of a crash.
A little later, it complains:
ib_mthca :05:00.0: SW2HW_MPT returned status 0x0a
which doesn't sound quite as good... and things are very hosed
from that moment on; reloading ib_mthca seems to fix things, however.
Olaf
--
Olaf Kirch | --- o --- Nous sommes du soleil we love when we play
[EMAIL PROTECTED] |/ | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax
--- snip ---
From: Olaf Kirch [EMAIL PROTECTED]
Subject: Return proper error codes from mthca_fmr_alloc
If the allocation of the MTT or the mailbox failed, mthca_fmr_alloc
would return 0 (success) no matter what. This leads to crashes a little
down the road, when we try to dereference eg mr-mtt, which was
really ERR_PTR(-ENOMEM).
Signed-off-by: Olaf Kirch [EMAIL PROTECTED]
---
drivers/infiniband/hw/mthca/mthca_mr.c |8 ++--
1 file changed, 6 insertions(+), 2 deletions(-)
Index: ofa_kernel-1.3/drivers/infiniband/hw/mthca/mthca_mr.c
===
--- ofa_kernel-1.3.orig/drivers/infiniband/hw/mthca/mthca_mr.c
+++ ofa_kernel-1.3/drivers/infiniband/hw/mthca/mthca_mr.c
@@ -613,8 +613,10 @@ int mthca_fmr_alloc(struct mthca_dev *de
sizeof *(mr-mem.tavor.mpt) * idx;
mr-mtt = __mthca_alloc_mtt(dev, list_len, dev-mr_table.fmr_mtt_buddy);
- if (IS_ERR(mr-mtt))
+ if (IS_ERR(mr-mtt)) {
+ err = PTR_ERR(mr-mtt);
goto err_out_table;
+ }
mtt_seg = mr-mtt-first_seg * MTHCA_MTT_SEG_SIZE;
@@ -627,8 +629,10 @@ int mthca_fmr_alloc(struct mthca_dev *de
mr-mem.tavor.mtts = dev-mr_table.tavor_fmr.mtt_base + mtt_seg;
mailbox = mthca_alloc_mailbox(dev, GFP_KERNEL);
- if (IS_ERR(mailbox))
+ if (IS_ERR(mailbox)) {
+ err = PTR_ERR(mailbox);
goto err_out_free_mtt;
+ }
mpt_entry = mailbox-buf;
___
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] ofed-1.3-rc4 IB/ehca: Alloc firmware context in query_port() with GFP_ATOMIC
Hello Tziporet and Vlad! This is the small bug fix I mentioned in the call today. Please approve it for rc4. Thanks Nam ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Your profile
Hello! I am tired today. I am nice girl that would like to chat with you. Email me at [EMAIL PROTECTED] only, because I am using my friend's email to write this. To see my pics ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] OFED meeting agenda on 1.3-rc3 status and rc4 readiness
This is the agenda to OFED meeting today on 1.3-rc3 status and rc4 readiness Reminder for the release schedule * RC3 - done (30-Jan) * RC4 - Feb 6 or 7 * RC5 - Feb 18 == Gold (is this a vacation day in US?) * GA - Feb 25 Agenda: 1. Status update - all 2. Agree on the above schedule 3. Critical/Major bugs review: 874 critical[EMAIL PROTECTED] Intel MPI (IMB test) hangs intermittently on the qlogic HCA 846 critical[EMAIL PROTECTED] SDP crash on RHEL5 ppc64 running netserver 878 critical[EMAIL PROTECTED] slow failover with bonding and connected mode 888 critical[EMAIL PROTECTED] OSU latency benchmark (old version with iteration and message size parameter) stuck sometime 887 critical[EMAIL PROTECTED] IMB benchmark stuck 760 major [EMAIL PROTECTED] UDP performance on Rx is lower than Tx 736 major [EMAIL PROTECTED] IBV_WC_RETRY_EXC_ERR errors with local rdma_reads Tziporet ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] IPoIB-UD 4K MTU patch for RC3 against 2.6.24
Hello Vlad,
Here is the IPoIB-4K MTU patch for OFED-1.3-RC3 release agains 2.6.24
kernel. I create an attachment as well since my email has some problem.
Regarding the backport, one line is needed to add for priv-stats vs.
dev-stats. I don't have the backport patch, if you could help me that
would be nice. If this is any issue, I will ask Nam to help out.
I have touch tested mthca for 2K MTU for the updated patch. More test
are going on.
thanks
Shirley
Signed-off-by Shirley Ma [EMAIL PROTECTED]
---
drivers/infiniband/ulp/ipoib/ipoib.h | 28 +++-
drivers/infiniband/ulp/ipoib/ipoib_ib.c| 218
+---
drivers/infiniband/ulp/ipoib/ipoib_main.c | 19 ++-
drivers/infiniband/ulp/ipoib/ipoib_multicast.c |3 +-
drivers/infiniband/ulp/ipoib/ipoib_verbs.c | 16 ++-
5 files changed, 212 insertions(+), 72 deletions(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h
b/drivers/infiniband/ulp/ipoib/ipoib.h
index 8eb6aa2..cb3aeab 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -56,11 +56,11 @@
/* constants */
enum {
- IPOIB_PACKET_SIZE = 2048,
- IPOIB_BUF_SIZE= IPOIB_PACKET_SIZE + IB_GRH_BYTES,
-
IPOIB_ENCAP_LEN = 4,
+ IPOIB_UD_HEAD_SIZE= IB_GRH_BYTES + IPOIB_ENCAP_LEN,
+ IPOIB_UD_RX_SG= 2, /* for 4K MTU */
+
IPOIB_CM_MTU = 0x1 - 0x10, /* padding to align header
to 16 */
IPOIB_CM_BUF_SIZE = IPOIB_CM_MTU + IPOIB_ENCAP_LEN,
IPOIB_CM_HEAD_SIZE= IPOIB_CM_BUF_SIZE % PAGE_SIZE,
@@ -135,9 +135,9 @@ struct ipoib_mcast {
struct net_device *dev;
};
-struct ipoib_rx_buf {
+struct ipoib_sg_rx_buf {
struct sk_buff *skb;
- u64 mapping;
+ u64 mapping[IPOIB_UD_RX_SG];
};
struct ipoib_tx_buf {
@@ -286,7 +286,7 @@ struct ipoib_dev_priv {
unsigned int admin_mtu;
unsigned int mcast_mtu;
- struct ipoib_rx_buf *rx_ring;
+ struct ipoib_sg_rx_buf *rx_ring;
spinlock_t tx_lock;
struct ipoib_tx_buf *tx_ring;
@@ -315,6 +315,9 @@ struct ipoib_dev_priv {
struct dentry *mcg_dentry;
struct dentry *path_dentry;
#endif
+ int max_ib_mtu;
+ struct ib_sge rx_sge[IPOIB_UD_RX_SG];
+ struct ib_recv_wr rx_wr;
};
struct ipoib_ah {
@@ -355,6 +358,19 @@ struct ipoib_neigh {
struct list_headlist;
};
+#define IPOIB_UD_MTU(ib_mtu) (ib_mtu - IPOIB_ENCAP_LEN)
+#define IPOIB_UD_BUF_SIZE(ib_mtu) (ib_mtu + IB_GRH_BYTES)
+static inline int ipoib_ud_need_sg(int ib_mtu)
+{
+ return (IPOIB_UD_BUF_SIZE(ib_mtu) PAGE_SIZE) ? 1 : 0;
+}
+static inline void ipoib_sg_dma_unmap_rx(struct ipoib_dev_priv *priv,
+u64 mapping[IPOIB_UD_RX_SG])
+{
+ ib_dma_unmap_single(priv-ca, mapping[0], IPOIB_UD_HEAD_SIZE,
DMA_FROM_DEVICE);
+ ib_dma_unmap_single(priv-ca, mapping[1], PAGE_SIZE, DMA_FROM_DEVICE);
+}
+
/*
* We stash a pointer to our private neighbour information after our
* hardware address in neigh-ha. The ALIGN() expression here makes
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
index 5063dd5..6c9eefe 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
@@ -87,32 +87,93 @@ void ipoib_free_ah(struct kref *kref)
spin_unlock_irqrestore(priv-lock, flags);
}
+/* Adjust length of skb with fragments to match received data */
+static void ipoib_ud_skb_put_frags(struct sk_buff *skb, unsigned int
length,
+ struct sk_buff *toskb)
+{
+ unsigned int size;
+ skb_frag_t *frag = skb_shinfo(skb)-frags[0];
+
+ /* put header into skb */
+ size = min(length, (unsigned)IPOIB_UD_HEAD_SIZE);
+ skb-tail += size;
+ skb-len += size;
+ length -= size;
+
+ if (length == 0) {
+ /* don't need this page */
+ skb_fill_page_desc(toskb, 0, frag-page, 0, PAGE_SIZE);
+ --skb_shinfo(skb)-nr_frags;
+ } else {
+ size = min(length, (unsigned) PAGE_SIZE);
+ frag-size = size;
+ skb-data_len += size;
+ skb-truesize += size;
+ skb-len += size;
+ length -= size;
+ }
+}
+
+static struct sk_buff *ipoib_sg_alloc_rx_skb(struct net_device *dev,
+int id, u64
mapping[IPOIB_UD_RX_SG])
+{
+ struct ipoib_dev_priv *priv = netdev_priv(dev);
+ struct page *page;
+ struct sk_buff *skb;
+
+ skb = dev_alloc_skb(IPOIB_UD_HEAD_SIZE);
+
+ if (unlikely(!skb))
+ return NULL;
+
+ mapping[0] = ib_dma_map_single(priv-ca, skb-data,
IPOIB_UD_HEAD_SIZE,
+ DMA_FROM_DEVICE);
+ if
[ewg] OFED teleconference today
Friendly reminder: the OFED teleconference is today (Monday, 4 February, 2008). Next few teleconferences: - All are at noon US eastern / 9am US Pacific / 7pm Israel 1. Monday, Feb 4, code 210020028 2. Monday, Feb 11, code 210020028 3. Monday, Feb 18, code 210020028 Dial-in information: US/Canada: +1.866.432.9903 India: +91.80.4103.3979 Israel: +972.9.892.7026 Others: http://cisco.com/en/US/about/doing_business/conferencing/ -- Jeff Squyres Cisco Systems ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
Re: [ewg] OFED meeting agenda on 1.3-rc3 status and rc4 readiness
Tziporet, I cannot attend today's call. I have status below: Tziporet Koren wrote: This is the agenda to OFED meeting today on 1.3-rc3 status and rc4 readiness Reminder for the release schedule * RC3 - done (30-Jan) * RC4 - Feb 6 or 7 * RC5 - Feb 18 == Gold (is this a vacation day in US?) * GA - Feb 25 Agenda: 1. Status update - all Uncovered a cxgb3 bug that we need fixed for ofed-1.3. I just opened 890 to track this. I hope to have a fix today or tomorrow... Also, I posted a trivial change to rmda_lat to enable it on chelsio devices. This was an oversight that should have been fixed a while ago. 2. Agree on the above schedule Agree on the schedule, but I need bug 890 in. Most likely it'll have to go in RC5. steve. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] perftest tools
Who owns perftest these days? I have a small change to rdma_lat.c I'd like to consider for ofed-1.3. It adds a new option to allow specifying the max inline size, which is currently hard-coded to 400. 400 is too big for chelsio devices. Adding a command line option will allow the user to specify a different value. I have a test patch ready. Steve. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [PATCH] IPoIB-UD 4K MTU patch for RC3 against 2.6.24
Hello all, I have created the patch and tested without Eli's patch but with Pradeep's patch. It works OK. Then I create another patch with Eli and Pradeep's patch against today's ofed-1.3 git tree. The ping worked for a while then stopped. I will try to debug it. And We have found a crash in today's ofed git tree in IPoIB-CM mode. Pradeep has narrowed down it to Eli's patch. Please address it on time. So we can continue our test. thanks Shirley ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Ado6e Master Sui+e for XP|Vls+a 299, Retail 2499 (save 2199)
abbyy finereader 8.0 professional multilanguage - 49 adobe encore dvd 2 - 49 acronis true image workstation 9.1.3887 - 29 adobe indesign cs3 - 79 creative suite premium 2 - 149 adobe atmosphere 1.0 - 29 mindjet mindmanager pro 7.0 - 39 adobe illustrator cs2 - 59 sony acid pro 6 - 59 v!slt 2008adobedeals. com in Web Browser all were killed before earth was reached. It was just as well for See here, young man! he said when he saw what was happening, I and peering up. ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [PATCH] DAPL v2 - fix package to build against and target /dat2/include
Fix OFED v2 package to build against and target /dat2/include directory. Signed-off by: Arlin Davis [EMAIL PROTECTED] diff --git a/Makefile.am b/Makefile.am index c4cb1bf..60b3db6 100755 --- a/Makefile.am +++ b/Makefile.am @@ -183,17 +183,17 @@ dapl_udapl_libdaplofa_la_LDFLAGS = -version-info 2:0:0 $(daplofa_version_script) libdatincludedir = $(includedir)/dat2 -libdatinclude_HEADERS = dat/include/dat/dat.h \ - dat/include/dat/dat_error.h \ - dat/include/dat/dat_platform_specific.h \ - dat/include/dat/dat_redirection.h \ - dat/include/dat/dat_registry.h \ - dat/include/dat/dat_vendor_specific.h \ - dat/include/dat/udat_config.h \ - dat/include/dat/udat.h \ - dat/include/dat/udat_redirection.h \ - dat/include/dat/udat_vendor_specific.h \ - dat/include/dat/dat_ib_extensions.h +libdatinclude_HEADERS = dat/include/dat2/dat.h \ + dat/include/dat2/dat_error.h \ + dat/include/dat2/dat_platform_specific.h \ + dat/include/dat2/dat_redirection.h \ + dat/include/dat2/dat_registry.h \ + dat/include/dat2/dat_vendor_specific.h \ + dat/include/dat2/udat_config.h \ + dat/include/dat2/udat.h \ + dat/include/dat2/udat_redirection.h \ + dat/include/dat2/udat_vendor_specific.h \ + dat/include/dat2/dat_ib_extensions.h man_MANS = man/dtest.1 man/dapltest.1 @@ -203,16 +203,16 @@ EXTRA_DIST = dat/common/dat_dictionary.h \ dat/common/dat_sr.h \ dat/udat/udat_sr_parser.h \ dat/udat/linux/dat_osd.h \ -dat/include/dat/dat.h \ -dat/include/dat/dat_error.h \ -dat/include/dat/dat_platform_specific.h \ -dat/include/dat/dat_redirection.h \ -dat/include/dat/dat_registry.h \ -dat/include/dat/dat_vendor_specific.h \ -dat/include/dat/udat_config.h \ -dat/include/dat/udat.h \ -dat/include/dat/udat_redirection.h \ -dat/include/dat/udat_vendor_specific.h \ +dat/include/dat2/dat.h \ +dat/include/dat2/dat_error.h \ +dat/include/dat2/dat_platform_specific.h \ +dat/include/dat2/dat_redirection.h \ +dat/include/dat2/dat_registry.h \ +dat/include/dat2/dat_vendor_specific.h \ +dat/include/dat2/udat_config.h \ +dat/include/dat2/udat.h \ +dat/include/dat2/udat_redirection.h \ +dat/include/dat2/udat_vendor_specific.h \ dapl/common/dapl_adapter_util.h \ dapl/common/dapl_cno_util.h \ dapl/common/dapl_cookie.h \ diff --git a/dapl/ibal/dapl_ibal_dto.h b/dapl/ibal/dapl_ibal_dto.h index be5686f..283fd91 100644 --- a/dapl/ibal/dapl_ibal_dto.h +++ b/dapl/ibal/dapl_ibal_dto.h @@ -52,7 +52,7 @@ #include dapl_ibal_util.h #ifdef DAT_EXTENSIONS -#include dat/dat_ib_extensions.h +#include dat2/dat_ib_extensions.h #endif extern DAT_RETURN diff --git a/dapl/include/dapl.h b/dapl/include/dapl.h index 49a3f49..42db81e 100755 --- a/dapl/include/dapl.h +++ b/dapl/include/dapl.h @@ -45,11 +45,11 @@ #define _DAPL_H_ #if defined(__KERNEL__) -#include dat/kdat.h +#include dat2/kdat.h #else -#include dat/udat.h +#include dat2/udat.h #endif /* defined(__KERNEL__) */ -#include dat/dat_registry.h +#include dat2/dat_registry.h #include dapl_osd.h #include dapl_debug.h diff --git a/dapl/openib_cma/dapl_ib_dto.h b/dapl/openib_cma/dapl_ib_dto.h index 09cef14..cea989b 100644 --- a/dapl/openib_cma/dapl_ib_dto.h +++ b/dapl/openib_cma/dapl_ib_dto.h @@ -51,7 +51,7 @@ #include dapl_ib_util.h #ifdef DAT_EXTENSIONS -#include dat/dat_ib_extensions.h +#include dat2/dat_ib_extensions.h #endif #defineDEFAULT_DS_ENTRIES 8 diff --git a/dapl/udapl/dapl_init.c b/dapl/udapl/dapl_init.c index 94a4619..ce92f9f 100644 --- a/dapl/udapl/dapl_init.c +++ b/dapl/udapl/dapl_init.c @@ -37,7 +37,7 @@ **/ #include dapl.h -#include dat/dat_registry.h /* Provider API function prototypes */ +#include dat2/dat_registry.h /* Provider API function prototypes */ #include dapl_hca_util.h #include dapl_init.h #include dapl_provider.h diff --git a/dapl/udapl/linux/dapl_osd.h b/dapl/udapl/linux/dapl_osd.h index e932ae0..caf971f 100644 --- a/dapl/udapl/linux/dapl_osd.h +++ b/dapl/udapl/linux/dapl_osd.h @@ -54,7 +54,7 @@ #endif -#include dat/udat.h +#include dat2/udat.h #include assert.h #include errno.h #include pthread.h diff --git a/dat/common/dat_api.c b/dat/common/dat_api.c index
[ewg] [PATCH] IPoIB-UD 4K MTU patch against 2.6.24 ofed-1.3-git tree
Tziporet,
This IPoIB 4K MTU patch is built against today's 2.6.24 OFED-1.3-Git
tree. This patch tested before Eli's patch successfully. This rebuilt
patch is on top of Eli's patch. However this constant UD_POST_RCV_COUNT
which is defined in Eli's patch as 16 does impact the behavior this
patch. When I define this as 1, everything works OK, if I change the
value to 8 or bigger, the patch won't work well.
We do see a couple of issues after Eli's patch checks in. So I suggest
to check in the patch. Then we can work together to address these issues
tomorrow. In Eli's patch I would suggest use kzalloc() to alloc 16
ib_sge and ib_recv_wr instead of defining this in ipoib_dev_priv since
it might have some memory issue there. I am working on the patch now to
see any better results.
Vlad,
There would be one line change for backporting regarding priv-stats vs.
dev-stats. If you have any problem to create the backport patch, let me
know. I will ask Nam to help. The attachment is for you to easily apply
the patch, my email might have issues.
Thanks
Shirley
Signed-off-by: Shirley Ma [EMAIL PROTECTED]
---
diff -urpN ofed_1_3_a/drivers/infiniband/ulp/ipoib/ipoib.h
ofed_1_3_b/drivers/infiniband/ulp/ipoib/ipoib.h
--- ofed_1_3_a/drivers/infiniband/ulp/ipoib/ipoib.h 2008-02-04
15:45:44.0 -0800
+++ ofed_1_3_b/drivers/infiniband/ulp/ipoib/ipoib.h 2008-02-04
15:40:38.0 -0800
@@ -56,11 +56,11 @@
/* constants */
enum {
- IPOIB_PACKET_SIZE = 2048,
- IPOIB_BUF_SIZE= IPOIB_PACKET_SIZE + IB_GRH_BYTES,
-
IPOIB_ENCAP_LEN = 4,
+ IPOIB_UD_HEAD_SIZE= IB_GRH_BYTES + IPOIB_ENCAP_LEN,
+ IPOIB_UD_RX_SG= 2, /* for 4K MTU */
+
IPOIB_CM_MTU = 0x1 - 0x10, /* padding to align header
to 16 */
IPOIB_CM_BUF_SIZE = IPOIB_CM_MTU + IPOIB_ENCAP_LEN,
IPOIB_CM_HEAD_SIZE= IPOIB_CM_BUF_SIZE % PAGE_SIZE,
@@ -141,9 +141,9 @@ struct ipoib_mcast {
struct net_device *dev;
};
-struct ipoib_rx_buf {
+struct ipoib_sg_rx_buf {
struct sk_buff *skb;
- u64 mapping;
+ u64 mapping[IPOIB_UD_RX_SG];
};
struct ipoib_tx_buf {
@@ -337,7 +337,7 @@ struct ipoib_dev_priv {
struct net_device *dev;
struct ib_recv_wr rx_wr_draft[UD_POST_RCV_COUNT];
- struct ib_sge sglist_draft[UD_POST_RCV_COUNT];
+ struct ib_sge sglist_draft[UD_POST_RCV_COUNT][IPOIB_UD_RX_SG];
unsigned intrx_outst;
struct napi_struct napi;
@@ -378,7 +378,7 @@ struct ipoib_dev_priv {
unsigned int admin_mtu;
unsigned int mcast_mtu;
- struct ipoib_rx_buf *rx_ring;
+ struct ipoib_sg_rx_buf *rx_ring;
spinlock_t tx_lock;
struct ipoib_tx_buf *tx_ring;
@@ -412,6 +412,7 @@ struct ipoib_dev_priv {
struct ipoib_ethtool_st etool;
struct timer_list poll_timer;
struct ib_ah *own_ah;
+ int max_ib_mtu;
};
struct ipoib_ah {
@@ -452,6 +453,19 @@ struct ipoib_neigh {
struct list_headlist;
};
+#define IPOIB_UD_MTU(ib_mtu) (ib_mtu - IPOIB_ENCAP_LEN)
+#define IPOIB_UD_BUF_SIZE(ib_mtu) (ib_mtu + IB_GRH_BYTES)
+static inline int ipoib_ud_need_sg(int ib_mtu)
+{
+ return (IPOIB_UD_BUF_SIZE(ib_mtu) PAGE_SIZE) ? 1 : 0;
+}
+static inline void ipoib_sg_dma_unmap_rx(struct ipoib_dev_priv *priv,
+u64 mapping[IPOIB_UD_RX_SG])
+{
+ ib_dma_unmap_single(priv-ca, mapping[0], IPOIB_UD_HEAD_SIZE,
DMA_FROM_DEVICE);
+ ib_dma_unmap_single(priv-ca, mapping[1], PAGE_SIZE, DMA_FROM_DEVICE);
+}
+
/*
* We stash a pointer to our private neighbour information after our
* hardware address in neigh-ha. The ALIGN() expression here makes
diff -urpN ofed_1_3_a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
ofed_1_3_b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
--- ofed_1_3_a/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2008-02-04
15:45:44.0 -0800
+++ ofed_1_3_b/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2008-02-04
15:40:38.0 -0800
@@ -96,14 +96,82 @@ static void clean_pending_receives(struc
for (i = 0; i priv-rx_outst; ++i) {
id = priv-rx_wr_draft[i].wr_id ~IPOIB_OP_RECV;
- ib_dma_unmap_single(priv-ca, priv-rx_ring[id].mapping,
-IPOIB_BUF_SIZE,
DMA_FROM_DEVICE);
+ if (ipoib_ud_need_sg(priv-max_ib_mtu))
+ ipoib_sg_dma_unmap_rx(priv,
+ priv-rx_ring[i].mapping);
+ else
+ ib_dma_unmap_single(priv-ca,
priv-rx_ring[id].mapping[0],
+
IPOIB_UD_BUF_SIZE(priv-max_ib_mtu), DMA_FROM_DEVICE);
dev_kfree_skb_any(priv-rx_ring[id].skb);
priv-rx_ring[id].skb = NULL;
}
priv-rx_outst = 0;
}
+static void ipoib_ud_skb_put_frags(struct
Re: [ewg] Oops with today's OFED 1.3
Eli, Please look at this issues ASAP. Without your patch everything works well. Thanks Shirley ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
***SPAM*** Re: [ewg] Re: [ofa-general] Please send all patches for OFED 1.3 rc4 by end of Monday (Feb 4)
Tziporet Koren [EMAIL PROTECTED] wrote on 02/04/2008 08:14:08 AM: OK - go ahead and regenerate patch and we will be able to include it in RC4 BTW - how did you test it with mthca? It does not support 4K MTU. You can test it with ConnectX since it does supports 4K MTU (with a special burning configuration). Please let me know if you have ConnectX and you wish to test it with 4K MTU Tziporet Thanks Tzipoeret. I would like to test ConnectX. But I can't test right it now since the switch connected to ConnectX is configured as 2K MTU and the test team has other test task to finish. But I can suggest the test team to include 4K MTU test as port of their system validation. Please send me the instructions on how to enable it for ConnectX. Thanks Shirley___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [ANNOUCE] dapl 2.0.6 release
New release for dapl 2.0 available on the OFA download page and in my git tree. - changes to correctly build against and target /dat2/include directory. md5sum: c0ad3d8b5df252ff5ce47f616a462f4edapl-2.0.6.tar.gz Vlad, please pull new v2 release into OFED 1.3 RC4 and install the following dapl packages: dapl-1.2.4-1 dapl-devel-1.2.4-1 dapl-2.0.6-1 dapl-utils-2.0.6-1 dapl-devel-2.0.6-1 dapl-debuginfo-2.0.6-1 See blocked::http://www.openfabrics.org/downloads/dapl/README.html http://www.openfabrics.org/downloads/dapl/README.html for details. -arlin ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] [UPDATE][PATCH] IPoIB-UD 4K MTU patch against 2.6.24 ofed-1.3-git tree
I found one one line was out side for loop when merging this patch
with current git-tree. This caused UD_POST_RCV_COUNT = 16 wrong. I have
fixed it. This is the updated patch.
Thanks
Shirley
Signed-off-by: Shirley Ma [EMAIL PROTECTED]
---
diff -urpN ofed_kernel_a/drivers/infiniband/ulp/ipoib/ipoib.h
ofed_kernel_b/drivers/infiniband/ulp/ipoib/ipoib.h
--- ofed_kernel_a/drivers/infiniband/ulp/ipoib/ipoib.h 2008-02-04
20:09:18.0 -0800
+++ ofed_kernel_b/drivers/infiniband/ulp/ipoib/ipoib.h 2008-02-04
20:11:26.0 -0800
@@ -56,11 +56,11 @@
/* constants */
enum {
- IPOIB_PACKET_SIZE = 2048,
- IPOIB_BUF_SIZE= IPOIB_PACKET_SIZE + IB_GRH_BYTES,
-
IPOIB_ENCAP_LEN = 4,
+ IPOIB_UD_HEAD_SIZE= IB_GRH_BYTES + IPOIB_ENCAP_LEN,
+ IPOIB_UD_RX_SG= 2, /* for 4K MTU */
+
IPOIB_CM_MTU = 0x1 - 0x10, /* padding to align header
to 16 */
IPOIB_CM_BUF_SIZE = IPOIB_CM_MTU + IPOIB_ENCAP_LEN,
IPOIB_CM_HEAD_SIZE= IPOIB_CM_BUF_SIZE % PAGE_SIZE,
@@ -141,9 +141,9 @@ struct ipoib_mcast {
struct net_device *dev;
};
-struct ipoib_rx_buf {
+struct ipoib_sg_rx_buf {
struct sk_buff *skb;
- u64 mapping;
+ u64 mapping[IPOIB_UD_RX_SG];
};
struct ipoib_tx_buf {
@@ -337,7 +337,7 @@ struct ipoib_dev_priv {
struct net_device *dev;
struct ib_recv_wr rx_wr_draft[UD_POST_RCV_COUNT];
- struct ib_sge sglist_draft[UD_POST_RCV_COUNT];
+ struct ib_sge sglist_draft[UD_POST_RCV_COUNT][IPOIB_UD_RX_SG];
unsigned intrx_outst;
struct napi_struct napi;
@@ -378,7 +378,7 @@ struct ipoib_dev_priv {
unsigned int admin_mtu;
unsigned int mcast_mtu;
- struct ipoib_rx_buf *rx_ring;
+ struct ipoib_sg_rx_buf *rx_ring;
spinlock_t tx_lock;
struct ipoib_tx_buf *tx_ring;
@@ -412,6 +412,7 @@ struct ipoib_dev_priv {
struct ipoib_ethtool_st etool;
struct timer_list poll_timer;
struct ib_ah *own_ah;
+ int max_ib_mtu;
};
struct ipoib_ah {
@@ -452,6 +453,19 @@ struct ipoib_neigh {
struct list_headlist;
};
+#define IPOIB_UD_MTU(ib_mtu) (ib_mtu - IPOIB_ENCAP_LEN)
+#define IPOIB_UD_BUF_SIZE(ib_mtu) (ib_mtu + IB_GRH_BYTES)
+static inline int ipoib_ud_need_sg(int ib_mtu)
+{
+ return (IPOIB_UD_BUF_SIZE(ib_mtu) PAGE_SIZE) ? 1 : 0;
+}
+static inline void ipoib_sg_dma_unmap_rx(struct ipoib_dev_priv *priv,
+u64 mapping[IPOIB_UD_RX_SG])
+{
+ ib_dma_unmap_single(priv-ca, mapping[0], IPOIB_UD_HEAD_SIZE,
DMA_FROM_DEVICE);
+ ib_dma_unmap_single(priv-ca, mapping[1], PAGE_SIZE, DMA_FROM_DEVICE);
+}
+
/*
* We stash a pointer to our private neighbour information after our
* hardware address in neigh-ha. The ALIGN() expression here makes
diff -urpN ofed_kernel_a/drivers/infiniband/ulp/ipoib/ipoib_ib.c
ofed_kernel_b/drivers/infiniband/ulp/ipoib/ipoib_ib.c
--- ofed_kernel_a/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2008-02-04
20:09:18.0 -0800
+++ ofed_kernel_b/drivers/infiniband/ulp/ipoib/ipoib_ib.c 2008-02-04
20:11:26.0 -0800
@@ -96,14 +96,82 @@ static void clean_pending_receives(struc
for (i = 0; i priv-rx_outst; ++i) {
id = priv-rx_wr_draft[i].wr_id ~IPOIB_OP_RECV;
- ib_dma_unmap_single(priv-ca, priv-rx_ring[id].mapping,
-IPOIB_BUF_SIZE,
DMA_FROM_DEVICE);
+ if (ipoib_ud_need_sg(priv-max_ib_mtu))
+ ipoib_sg_dma_unmap_rx(priv,
+ priv-rx_ring[i].mapping);
+ else
+ ib_dma_unmap_single(priv-ca,
priv-rx_ring[id].mapping[0],
+
IPOIB_UD_BUF_SIZE(priv-max_ib_mtu), DMA_FROM_DEVICE);
dev_kfree_skb_any(priv-rx_ring[id].skb);
priv-rx_ring[id].skb = NULL;
}
priv-rx_outst = 0;
}
+static void ipoib_ud_skb_put_frags(struct sk_buff *skb, unsigned int
length,
+ struct sk_buff *toskb)
+{
+ unsigned int size;
+ skb_frag_t *frag = skb_shinfo(skb)-frags[0];
+
+ /* put header into skb */
+ size = min(length, (unsigned)IPOIB_UD_HEAD_SIZE);
+ skb-tail += size;
+ skb-len += size;
+ length -= size;
+
+ if (length == 0) {
+ /* don't need this page */
+ skb_fill_page_desc(toskb, 0, frag-page, 0, PAGE_SIZE);
+ --skb_shinfo(skb)-nr_frags;
+ } else {
+ size = min(length, (unsigned) PAGE_SIZE);
+ frag-size = size;
+ skb-data_len += size;
+ skb-truesize += size;
+ skb-len += size;
+ length -= size;
+ }
+}
+
+static struct sk_buff
Re: [ewg] [PATCH ofed-1.3] rdma_lat: Add -m --max-inline option to support devices with different
Hi Steve. Steve Wise wrote: Oren, I have a small change to rdma_lat.c I'd like to get in ofed-1.3. It adds a new option to allow specifying the max inline size, which is currently hard-coded to 400. 400 is too big for chelsio devices. Adding a command line option will allow the user to specify a different value. The other tests in the perftest folder support this kind of functionality as well -I, --inline_size=size max size of message to be sent in inline mode (default 400) Can you please change your patch that all of the performance tests looks the same? thanks Dotan ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
[ewg] Re: [GIT PULL] ~sashak/management.git
Sasha Khapyorsky wrote: Hi Vlad, Please pull recent ofed_1_3 branch of ~sashak/management.git. The changes are: Sasha Khapyorsky (1): infiniband-diags: perl scripts formatting Tim Meier (2): infiniband-diags: formatting some perlscripts opensm: diags better error checking for DR option Timothy A. Meier (1): opensm: diags bug fix in iblinkinfo.pl Yevgeny Kliteynik (2): opensm/osm_ucast_ftree.c: do load-leveling of non-CN routes opensm/osm_ucast_ftree.c: cosmetics Thanks, Sasha Done, Regards, Vladimir ___ ewg mailing list ewg@lists.openfabrics.org http://lists.openfabrics.org/cgi-bin/mailman/listinfo/ewg
