[openib-general] [PATCH] ib_verbs.h (Roland's branch): Add MAD pointer to send WR ud structure
ib_verbs.h (Roland's branch): Add MAD pointer to send WR ud structure
This is to allow the MAD layer to poke at the MAD header without using
bus_to_virt().
Index: ib_verbs.h
===
--- ib_verbs.h (revision 939)
+++ ib_verbs.h (working copy)
@@ -523,6 +523,7 @@
} atomic;
struct {
struct ib_ah *ah;
+ struct ib_mad *mad;
u32 remote_qpn;
u32 remote_qkey;
int timeout_ms; /* valid for MADs only */
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad.c: Fix request/response matching
On Wed, 2004-10-06 at 10:36, Hal Rosenstock wrote:
On Tue, 2004-10-05 at 16:34, Sean Hefty wrote:
if (solicited) {
/* Routing is based on high 32 bits of transaction ID of MAD
*/
- hi_tid = mad-mad_hdr.tid 32;
+ hi_tid = be32_to_cpu(mad-mad_hdr.tid.tid_field.hi_tid);
This shouldn't be necessary:
It is still necessary as the TID is in network endian in the MAD header
and CPU endian in hi_tid.
Sender of request (system 1):
mad.tid = (mad_agent.hi_tid 32) | user_tid;
send mad
The problem with this is that when this is done on a little endian
machine it shows up byte swapped on the network and not in network
endian.
So if hi_tid = 1 and user tid = 0x9abcdef0
then the transaction ID in the MAD is 0xf0debc9a0100
I don't think that is what we want.
The client needs to do one more step:
Sender of request (system 1):
mad.tid = cpu_to_be64((mad_agent.hi_tid 32) | user_tid);
send mad
If this seems right, I can post a patch and remove the TID union.
Receiver of response (system 1):
hi_tid = mad.tid 32
-- Hal
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] ib_verbs.h (Roland's branch): Add MAD pointer to send WR ud structure
thanks, applied. ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad.c: Fix request/response matching
From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 7:36 AM The problem with this is that when this is done on a little endian machine it shows up byte swapped on the network and not in network endian. So if hi_tid = 1 and user tid = 0x9abcdef0 then the transaction ID in the MAD is 0xf0debc9a0100 I don't think that is what we want. The TID on the wire is opaque to any recipient. A response to a MAD should have exactly the same TID. The recipient of the response (original client) will then correctly decode the hi_tid and user_tid since the transaction ID is received exactly how it was sent. You only need byte swapping if the value needs to be interpreted by some other node with unknown endianness. If the recipient just blindly echoes the value back, the TID is effectively just a 64-bit data blob that it cares not about - byte ordering doesn't matter one bit. The endianness of the TID does not change for the client - it is sent in host order, and is received in host order. The only time this could cause problems is if your client's CPU changes endianness between the time the request is sent and the response received. I don't think we should bother coding for that possibility. - Fab ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ib_mad.h: Remove network endian conversion of QP1 QKey
ib_mad.h: Remove network endian conversion of QP1 QKey
Index: ib_mad.h
===
--- ib_mad.h(revision 935)
+++ ib_mad.h(working copy)
@@ -58,7 +58,7 @@
#define IB_QP0 0
#define IB_QP1 cpu_to_be32(1)
-#define IB_QP1_QKEYcpu_to_be32(0x8001)
+#define IB_QP1_QKEY0x8001
struct ib_grh {
u32 version_tclass_flow;
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad: Add MAD pointer to send WR udstructure
On Wed, 2004-10-06 at 12:01, Sean Hefty wrote:
Index: include/ib_verbs.h
===
--- include/ib_verbs.h (revision 935)
+++ include/ib_verbs.h (working copy)
@@ -539,6 +539,7 @@
} atomic;
struct {
struct ib_ah *ah;
+struct ib_mad *mad;
I _think_ that the user only needs to reference something like the
following:
union ib_mad_hdrs {
struct ib_mad_hdr mad_hdr;
struct {
struct ib_mad_hdr mad_hdr;
struct ib_rmpp_hdr rmpp_hdr;
} hdr;
};
I don't think that we need a pointer to the MAD data.
OK. But your diffs are ahead of the tree right now so I can't change it
to union ib_mad_hdrs yet.
-- Hal
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] ib_mad.h: Remove network endian conversion of QP1 QKey
On Wed, 2004-10-06 at 12:42, Hal Rosenstock wrote: ib_mad.h: Remove network endian conversion of QP1 QKey I forgot to say that with this change GMPs including request/response matching is now working :-) -- Hal ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad.h: Remove network endian conversionof QP1 QKey
From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 9:42 AM ib_mad.h: Remove network endian conversion of QP1 QKey I don't get the point. Is IB_QP1_QKEY going to be treated in host order, and then swapped by someone at some point when posting? Does mthca expect the QKey to be provided in host order and then swap it when formatting the WQEs? - Fab ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [ANNOUNCE]: MAD layer with SMI/SMA
This is the formal announcement on the availability of the first release of the OpenIB MAD layer with SMI/SMA (located in the openib-candidate branch). The OpenIB access layer is comprised of the MAD layer and the SMI/SMA. These are currently implemented as separate modules. These are layered on top of the mthca driver and cannot run currently with the MAD layer from the roland_merge branch. These directions are how to combine the two for building and running. Note also that the stack currently ends here (e.g. don't expect to run IPoIB right now, although we are heading to this; especially don't expect to run SRP or SDP or u/kDAPL right now either). There is a README in src/linux-kernel/infiniband/access which also contains building and using instructions. I have now done the following since last night: Run through install and build process from scratch. Add virtual address to the ud parameter in ib_mad_send. Redo MAD completion handler error handling. Investigate TID endian. Initial GMP testing (including request/response) There is one patch pending relative to TID endian (and elimination of the TID union). Other Pending Items (that don't affect release): Trim method mask used in ib_smi.c There is a workaround on the hop pointer in ib_smi.c which needs a real fix. Undo commenting out of ib_mad_send in mthca_mad.c More GMP testing Change send WR ud parameter from MAD to MAD header ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad: Add MAD pointer to send WRudstructure
On Wed, 2004-10-06 at 12:47, Sean Hefty wrote:
I _think_ that the user only needs to reference something like the
following:
union ib_mad_hdrs {
struct ib_mad_hdr mad_hdr;
struct {
struct ib_mad_hdr mad_hdr;
struct ib_rmpp_hdr rmpp_hdr;
} hdr;
};
I don't think that we need a pointer to the MAD data.
OK. But your diffs are ahead of the tree right now so I can't change it
to union ib_mad_hdrs yet.
I don't actually have this in my tree.
Thought you might have started on RMPP :-)
Just thinking whether the user needs
to provide a pointer to just the headers, or to the entire MAD. The API
should be clear what the expectations are on the client.
I'll generate another patch for this.
-- Hal
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad.h: Remove network endian conversionof QP1 QKey
On Wed, 2004-10-06 at 12:46, Fab Tillier wrote: From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 9:42 AM ib_mad.h: Remove network endian conversion of QP1 QKey I don't get the point. Is IB_QP1_QKEY going to be treated in host order, and then swapped by someone at some point when posting? Does mthca expect the QKey to be provided in host order and then swap it when formatting the WQEs? In ib_mad.c, it's used as a QKey attribute to the HCA when the QP is initialized. In a client, it's used as the remote QKey in the send WR ud request. The only byte swapping is done by the HCA not in the MAD layer or client. -- Hal ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] using VAPI and CMAPI in kernel module.
On Tue, 2004-10-05 at 14:46 +0200, von Wyl wrote: Hi, I'm trying to compile a kernel module which use the VAPI and (maybe) the CMAPI. And when I compile a file with an inclusion of vapi.h I got this : Please use the new mthca driver instead of the old mellanox driver. -tduffy -- When they took the 4th Amendment, I was quiet because I didn't deal drugs. When they took the 6th Amendment, I was quiet because I am innocent. When they took the 2nd Amendment, I was quiet because I don't own a gun. Now they have taken the 1st Amendment, and I can only be quiet. --Lyle Myhr signature.asc Description: This is a digitally signed message part ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad.h: Remove network endianconversionof QP1 QKey
On Wed, 2004-10-06 at 12:46, Fab Tillier wrote: From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 9:42 AM ib_mad.h: Remove network endian conversion of QP1 QKey I don't get the point. Is IB_QP1_QKEY going to be treated in host order, and then swapped by someone at some point when posting? Does mthca expect the QKey to be provided in host order and then swap it when formatting the WQEs? In ib_mad.c, it's used as a QKey attribute to the HCA when the QP is initialized. In a client, it's used as the remote QKey in the send WR ud request. The only byte swapping is done by the HCA not in the MAD layer or client. Mthca performs byte-swapping on the qkey in both of these cases. At this point, I think the openib stack needs byte-ordering reworked, but it can probably wait. ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [ANNOUNCE]: MAD layer with SMI/SMA
Great! I will start merging this onto my tree as soon as I get my IPoIB driver into shape to commit. (I've currently torn it apart getting rid of the pseudo-ethernet layer, and I have it mostly working again). - Roland ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ib_mad: Eliminate ib_tid union
ib_mad: Eliminate ib_tid union
Index: access/ib_mad_priv.h
===
--- access/ib_mad_priv.h(revision 942)
+++ access/ib_mad_priv.h(working copy)
@@ -119,7 +119,7 @@
struct list_head agent_send_list;
struct ib_mad_agent *agent;
u64 wr_id; /* client WR ID */
- union ib_tid tid;
+ u64 tid;
int timeout_ms;
int refcount;
enum ib_wc_status status;
Index: access/ib_mad.c
===
--- access/ib_mad.c (revision 940)
+++ access/ib_mad.c (working copy)
@@ -344,7 +344,7 @@
return -ENOMEM;
}
- mad_send_wr-tid.id = send_wr-wr.ud.mad-mad_hdr.tid.id;
+ mad_send_wr-tid = send_wr-wr.ud.mad-mad_hdr.tid;
mad_send_wr-agent = mad_agent;
mad_send_wr-timeout_ms = cur_send_wr-wr.ud.timeout_ms;
if (mad_send_wr-timeout_ms)
@@ -682,7 +682,7 @@
/* Whether MAD was solicited determines type of routing to MAD client
*/
if (solicited) {
/* Routing is based on high 32 bits of transaction ID of MAD */
- hi_tid = be32_to_cpu(mad-mad_hdr.tid.tid_field.hi_tid);
+ hi_tid = be64_to_cpu(mad-mad_hdr.tid) 32;
list_for_each_entry(entry, port_priv-agent_list, agent_list) {
if (entry-agent.hi_tid == hi_tid) {
mad_agent = entry;
@@ -764,7 +764,7 @@
list_for_each_entry(mad_send_wr, mad_agent_priv-send_list,
agent_send_list) {
- if (mad_send_wr-tid.id == tid) {
+ if (mad_send_wr-tid == tid) {
/* Verify request is still valid */
if (mad_send_wr-status == IB_WC_SUCCESS
mad_send_wr-timeout_ms)
@@ -793,7 +793,7 @@
if (solicited) {
spin_lock_irqsave(mad_agent_priv-send_list_lock, flags);
mad_send_wr = find_send_req(mad_agent_priv,
- recv-mad.mad.mad_hdr.tid.id);
+ recv-mad.mad.mad_hdr.tid);
if (!mad_send_wr) {
spin_unlock_irqrestore(mad_agent_priv-send_list_lock,
flags);
Index: include/ib_mad.h
===
--- include/ib_mad.h(revision 943)
+++ include/ib_mad.h(working copy)
@@ -69,14 +69,6 @@
union ib_giddgid;
} __attribute__ ((packed));
-union ib_tid {
- u64 id;
- struct {
- u32 hi_tid;
- u32 lo_tid;
- } tid_field;
-};
-
struct ib_mad_hdr {
u8 base_version;
u8 mgmt_class;
@@ -84,7 +76,7 @@
u8 method;
u16 status;
u16 class_specific;
- union ib_tid tid;
+ u64 tid;
u16 attr_id;
u16 resv;
u32 attr_mod;
Index: include/ib_smi.h
===
--- include/ib_smi.h(revision 935)
+++ include/ib_smi.h(working copy)
@@ -41,7 +41,7 @@
u16 status;
u8 hop_ptr;
u8 hop_cnt;
- union ib_tid tid;
+ u64 tid;
u16 attr_id;
u16 resv;
u32 attr_mod;
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] ib_mad.c: Fix request/response matching
On Wed, 2004-10-06 at 12:04, Fab Tillier wrote: From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Wednesday, October 06, 2004 7:36 AM The problem with this is that when this is done on a little endian machine it shows up byte swapped on the network and not in network endian. So if hi_tid = 1 and user tid = 0x9abcdef0 then the transaction ID in the MAD is 0xf0debc9a0100 I don't think that is what we want. The TID on the wire is opaque to any recipient. True but by conforming to network endian it makes it easier to understand what is going on (which internal client should receive it). This is a minor cost. A response to a MAD should have exactly the same TID. The recipient of the response (original client) will then correctly decode the hi_tid and user_tid since the transaction ID is received exactly how it was sent. You only need byte swapping if the value needs to be interpreted by some other node with unknown endianness. If the recipient just blindly echoes the value back, the TID is effectively just a 64-bit data blob that it cares not about - byte ordering doesn't matter one bit. The endianness of the TID does not change for the client - it is sent in host order, and is received in host order. The only time this could cause problems is if your client's CPU changes endianness between the time the request is sent and the response received. I don't think we should bother coding for that possibility. Agreed. -- Hal ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ib_mad: Use pointer to madhdr rather than mad in send WR ud structure
ib_mad: Use pointer to madhdr rather than mad in send WR ud structure
We'll go the rest of the way on this when RMPP is implemented.
Index: access/ib_smi.c
===
--- access/ib_smi.c (revision 942)
+++ access/ib_smi.c (working copy)
@@ -361,7 +361,7 @@
send_wr.wr.ud.ah = ah;
send_wr.wr.ud.remote_qkey = 0; /* for SMPs */
- send_wr.wr.ud.mad = smp;
+ send_wr.wr.ud.mad_hdr = (struct ib_mad_hdr *)smp;
send_wr.wr_id = ++port_priv-wr_id;
pci_unmap_addr_set(smp, smi_send_wr-mapping, gather_list.addr);
Index: access/ib_mad.c
===
--- access/ib_mad.c (revision 944)
+++ access/ib_mad.c (working copy)
@@ -344,7 +344,7 @@
return -ENOMEM;
}
- mad_send_wr-tid = send_wr-wr.ud.mad-mad_hdr.tid;
+ mad_send_wr-tid = send_wr-wr.ud.mad_hdr-tid;
mad_send_wr-agent = mad_agent;
mad_send_wr-timeout_ms = cur_send_wr-wr.ud.timeout_ms;
if (mad_send_wr-timeout_ms)
Index: include/ib_verbs.h
===
--- include/ib_verbs.h (revision 940)
+++ include/ib_verbs.h (working copy)
@@ -539,7 +539,7 @@
} atomic;
struct {
struct ib_ah *ah;
- struct ib_mad *mad;
+ struct ib_mad_hdr *mad_hdr;
u32 remote_qpn;
u32 remote_qkey;
int timeout_ms; /* valid for MADs only */
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ib_verbs.h (Roland's branch): Use pointer to madhdr rather than mad in send WR ud structure
ib_verbs.h: Use pointer to madhdr rather than mad in send WR ud
structure
We'll go the rest of the way on this when RMPP is implemented.
(This patch applies to Roland's branch).
Index: ib_verbs.h
===
--- ib_verbs.h (revision 944)
+++ ib_verbs.h (working copy)
@@ -523,7 +523,7 @@
} atomic;
struct {
struct ib_ah *ah;
- struct ib_mad *mad;
+ struct ib_mad_hdr *mad_hdr;
u32 remote_qpn;
u32 remote_qkey;
int timeout_ms; /* valid for MADs only */
___
openib-general mailing list
[EMAIL PROTECTED]
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] mthca support for Arbel
Hi, Has mthca been tested with Arbel (PCI Express) ? Is this in compatibility mode or native mode or both ? Thanks. -- Hal ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mthca support for Arbel
Hal Hi, Has mthca been tested with Arbel (PCI Express) ? Is this Hal in compatibility mode or native mode or both ? I've tested mthca with Arbel in compatibility mode. I haven't had access to native mode firmware or up-to-date documentation, so I have not even started on native mode support. - Roland ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] for review to timeout send MADs
Here are some modifications to support timing out send MADs in the access layer. I
haven't tested this code beyond building it, but wanted to make it available for
review. There are a few race conditions that need to be avoided when handling
timeouts, so if it looks like something was missed, let me know.
Hal, can you either send me your test code or check it into svn somewhere? I want to
verify that this doesn't break your current tests, then expand the tests to check that
the timeout code is working properly.
Additional comments:
A couple of structure elements were renamed to better reflect their new usage.
mad_agents now have two lists: send_list and wait_list. mad_send_wr on the send_list
have active work requests posted for them. Once all work requests have completed, if
the mad_send_wr has a timeout and hasn't been canceled, it is moved to the wait_list.
A workqueue is used to schedule delayed processing of MAD timeouts. The scheduling
delay of the timeout thread is adjusted when a send completes or is canceled. If
anyone sees an issue with my usage of the workqueue, just let me know.
- Sean
-- Index: access/ib_mad_priv.h
===
--- access/ib_mad_priv.h(revision 946)
+++ access/ib_mad_priv.h(working copy)
@@ -58,6 +58,8 @@
#include linux/pci.h
#include linux/kthread.h
+#include linux/timer.h
+#include linux/workqueue.h
#include ib_mad.h
#include ib_smi.h
@@ -106,8 +108,11 @@
struct ib_mad_reg_req *reg_req;
struct ib_mad_port_private *port_priv;
- spinlock_t send_list_lock;
+ spinlock_t lock;
struct list_head send_list;
+ struct list_head wait_list;
+ struct work_struct work;
+ unsigned long timeout;
atomic_t refcount;
wait_queue_head_t wait;
@@ -116,11 +121,11 @@
struct ib_mad_send_wr_private {
struct list_head send_list;
- struct list_head agent_send_list;
+ struct list_head agent_list;
struct ib_mad_agent *agent;
u64 wr_id; /* client WR ID */
u64 tid;
- int timeout_ms;
+ unsigned long timeout;
int refcount;
enum ib_wc_status status;
};
Index: access/ib_mad.c
===
--- access/ib_mad.c (revision 946)
+++ access/ib_mad.c (working copy)
@@ -87,6 +87,7 @@
static void cancel_mads(struct ib_mad_agent_private *mad_agent_priv);
static void ib_mad_complete_send_wr(struct ib_mad_send_wr_private *mad_send_wr,
struct ib_mad_send_wc *mad_send_wc);
+static void timeout_sends(void *data);
/*
* ib_register_mad_agent - Register to send/receive MADs
@@ -225,8 +226,10 @@
list_add_tail(mad_agent_priv-agent_list, port_priv-agent_list);
spin_unlock_irqrestore(port_priv-reg_lock, flags);
- spin_lock_init(mad_agent_priv-send_list_lock);
+ spin_lock_init(mad_agent_priv-lock);
INIT_LIST_HEAD(mad_agent_priv-send_list);
+ INIT_LIST_HEAD(mad_agent_priv-wait_list);
+ INIT_WORK(mad_agent_priv-work, timeout_sends, mad_agent_priv);
atomic_set(mad_agent_priv-refcount, 1);
init_waitqueue_head(mad_agent_priv-wait);
mad_agent_priv-port_priv = port_priv;
@@ -254,17 +257,26 @@
mad_agent_priv = container_of(mad_agent, struct ib_mad_agent_private,
agent);
- /* Cleanup pending receives for this agent !!! */
+ /* Note that we could still be handling received MADs. */
+
+ /*
+* Canceling all sends results in dropping received response MADs,
+* preventing us from queuing additional work.
+*/
cancel_mads(mad_agent_priv);
+ cancel_delayed_work(mad_agent_priv-work);
+ flush_scheduled_work();
+
spin_lock_irqsave(mad_agent_priv-port_priv-reg_lock, flags);
remove_mad_reg_req(mad_agent_priv);
list_del(mad_agent_priv-agent_list);
spin_unlock_irqrestore(mad_agent_priv-port_priv-reg_lock, flags);
+
+ /* Cleanup pending RMPP receives for this agent !!! */
atomic_dec(mad_agent_priv-refcount);
- wait_event(mad_agent_priv-wait,
- !atomic_read(mad_agent_priv-refcount));
+ wait_event(mad_agent_priv-wait, !atomic_read(mad_agent_priv-refcount));
if (mad_agent_priv-reg_req)
kfree(mad_agent_priv-reg_req);
@@ -346,19 +358,19 @@
mad_send_wr-tid = send_wr-wr.ud.mad_hdr-tid;
mad_send_wr-agent = mad_agent;
- mad_send_wr-timeout_ms = cur_send_wr-wr.ud.timeout_ms;
- if (mad_send_wr-timeout_ms)
- mad_send_wr-refcount = 2;
- else
- mad_send_wr-refcount = 1;
+ /* Timeout will be updated after send completes. */
+ mad_send_wr-timeout = msecs_to_jiffies(
RE: [openib-general] mthca support for Arbel
Hal Hi, Has mthca been tested with Arbel (PCI Express) ? Is this in compatibility mode or native mode or both ? Thanks. -- Hal Does the PCI-Express HCA require any software changes to the HCA driver ? i.e., will it run with an existing PCI-X tavor driver ? I have heard that it does not require any changes, but I just wanted to confim that. ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mthca support for Arbel
Robert Does the PCI-Express HCA require any software changes to Robert the HCA driver ? i.e., will it run with an existing PCI-X Robert tavor driver ? I have heard that it does not require any Robert changes, but I just wanted to confim that. As Hal mentioned, there are two modes that the Mellanox PCIe HCA can run in: compatible and native mode. In compatible mode, the same driver as for the PCI-X HCA can be used. In native mode, new features are exposed (memory-free mode, verbs extensions, etc), but new software is required. - Roland ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [PATCH] for review to timeout send MADs
It seems you are using the system-wide keventd queue. This isn't necessarily a problem per se, but it would probably be better to use a MAD-layer private workqueue (I suggested a single-threaded workqueue per MAD port earlier). This avoids two problems. First, keventd is subject to arbitrary delays because it is used as a dumping ground for any code that needs to sleep. Second, if the MAD layer has its own workqueue, then it can be used for completion processing as well; as it stands it seems sort of funny to create a kthread to do some work and run other work from a workqueue. I am using the system level queue. If we think that using our own MAD queue is better, I will do that. I was thinking more along the lines of a single workqueue for all MAD services, with one per processor, rather than a workqueue per port, however. I was planning on changing completion processing to using work queues in a separate patch. A few low-level comments too: rbuf = list_entry(port_priv-recv_posted_mad_list[qpn], - struct ib_mad_recv_buf, - list); - rbuf = (struct ib_mad_recv_buf *)rbuf-list.next; +struct ib_mad_recv_buf, +list); + rbuf = (struct ib_mad_recv_buf*)rbuf-list.next; I don't understand what's going on here; can this not be written as: rbuf = list_entry(port_priv-recv_posted_mad_list[qpn].next, struct ib_mad_recv_buf, list); What happened is that I noticed that list_entry was being used like you saw and went to correct it the way you suggested, but backed out that change (manually) after I saw the problem in another area to avoid combining patches. I missed that I didn't revert to the original code. I've started a separate patch to fix-up this issue, and will make sure that this patch does not modify this area of the code. This patch seems whitespace-challenged in other places too: I'll go back and update this. Thanks. If you see other places that I missed other than what you mentioned, please let me know. ___ openib-general mailing list [EMAIL PROTECTED] http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
