Michael S. Tsirkin wrote:
Hmm.
BTW, we need to add something for userspace?
Userspace can already get at GIDs, I think, but how does it get the
IPoIB pkey?
Something needs to be done for userspace, but I'm not entirely sure what yet.
I've given it some thought, but was deferring doing too
properly. Once we call dma_map_single() on a buffer, the CPU may not
touch that buffer until after the corresponding dma_unmap_single().
It sounds like we need to change how the mapping is done. Can we let the MAD
layer always control the mapping? Considering how RMPP works, I'm not sure what
Roland Dreier wrote:
We probably still want to handle gather lists for posting sends I
think. Another (rather unrelated) issue that I just noticed the other
day is that something like sending a response to a GetTable request
for PortInfo for every port in a large fabric is going to end up
The following patch permits listening on a port number only.
All connection requests received on any RDMA device for that port
number are routed to the listening client.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: core/cma.c
Caitlin Bestler wrote:
No, I think Mike's comment was dead on. Applications want to
use the existing API. They want to use the existing API even
when the API is clearly defective. Note that there are several
generations of host-resolution APIs for the IP world, with the
earlier ones clearly
The following patch returns the GID of the IP gateway for non-local
subnet IP addresses.
Hal, does this change look correct to you? I don't have an easy way
to test this fully.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: core/addr.c
Roland Dreier wrote:
Thanks, I'll read this over.
What's the motivation here? To shift over to ib_create_send_mad() so
that all the MAD-related DMA mapping stuff is in one place, to make it
easier to fix?
Yes - the motivation is to fix the DMA mapping issue that you pointed out by
changing
Christoph Hellwig wrote:
If you change behaviour you should change the interface, in this case
you'd _really_ want to pass down the buffer as void pointer and not cast
it to a dma_addr_t - that would in fact break on ppc64 where dma_addr_t
is a 32bit data type and a pointer is 64bits wide.
To
Sean Hefty wrote:
+ query-sa_query.mad_buf = ib_create_send_mad(agent, 1, 0,
+query-sa_query.sm_ah-ah,
+ 0, IB_MGMT_MAD_DATA -
+ IB_MGMT_SA_DATA,
+IB_MGMT_SA_DATA
Here's an updated version.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: sa_query.c
===
--- sa_query.c (revision 3692)
+++ sa_query.c (working copy)
@@ -74,9 +74,8 @@ struct ib_sa_query {
void (*callback)(struct
Suresh Shelvapille wrote:
While writing a switch driver, I noticed that the alloc_pd and create_cq
function signatures are different depending on rdma/ib_verbs.h vs.
ib_verbs.h. I don't need RDMA for now, so going with the func signature as
in ib_verbs.h is OK or is there a necessity to use
Hal Rosenstock wrote:
Looks good. One comment below on agent_send_response.
Have you tested this ?
I did run with this change (along with the related changes to mthca and
sa_query). I was able to bring up the node, run ipoib, cmtest, and cmatose.
+void agent_send_response(struct ib_mad
Steve Wooding wrote:
I'm trying to make a QP connection using the CM, but the active side
cannot get to the RTR state. ibv_modify_qp returns errorno -22, invalid
argument.
How are you setting the QP attributes? You can try using the
ib_cm_init_qp_attr() call to set the attributes if you're
Hal Rosenstock wrote:
Yes, but why not ? (I think that was also part of your change).
The agent code now allocates a buffer for the MAD by calling
ib_create_send_mad(). The input MAD buffer is copied to the send buffer, then
transmitted. The result is that the input MAD buffer is always
Here's an updated version that defines agent_send_response as returning an int.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: agent.c
===
--- agent.c (revision 3794)
+++ agent.c (working copy)
@@ -36,58 +36,41
Tom Tucker wrote:
At first blush, the API looks good to me. The kinds of changes I was
pondering were related to hiding some of the routing issues. For
example, if the app. doesn't bind the rdma_cm_id prior to calling
rdma_connect, the code will lookup and use the default route instead of
Hal Rosenstock wrote:
The agent code now allocates a buffer for the MAD by calling
ib_create_send_mad(). The input MAD buffer is copied to the send buffer, then
transmitted. The result is that the input MAD buffer is always available for
posting on the receive queue.
What about the other
Hal Rosenstock wrote:
I was referring to the calls in agent_send_response which could fail.
It may not be obvious by reading the patch, but agent_send_response() now
consists of:
ib_get_agent_port()
ib_create_ah_from_wc()
ib_create_send_mad()
ib_post_send_mad()
The wc and grh params are
Kanevsky, Arkady wrote:
Enclosed is the proposal to IBTA to add this functionality to CM
protocol.
The main issue is that there is no protocol that provides
both src and dest IP addresses and ports and
provide 64 bytes of private
data to users simultaneously.
The last slide outlines 3
Kanevsky, Arkady wrote:
CM passes IB addresses of both src and dest in REQ.
How locally dest IP address is mapped to dest IB GID|LID is
defined by IPoIB.
We can request IBTA to define it also.
But the goal is to define a protocol part in IBTA.
The mapping from an IP address to a GID is
Kanevsky, Arkady wrote:
uDAPL users.
I'm not sure how much we should care about higher level abstractions for this
discussion. We should do what's right for IB. Abstractions that want to use IP
addresses can either use the standard protocol defined by the IBTA or define
their own private
I'm working on a patch to fix the DMA mapping issues that Roland reported
earlier on the mail list. The proposed solution involves the following
changes to ib_mad.h and ib_verbs.h. DMA mapping is performed immediately
before posting the work request, with unmapping coming after polling
the
Roland Dreier wrote:
struct ib_mad_send_buf {
+struct ib_mad_send_buf *next;
struct ib_mad *mad;
-DECLARE_PCI_UNMAP_ADDR(mapping)
Do we want to get rid of this field? It seems like we'll need to keep
track of the DMA mapping somewhere, and
Steven Wooding wrote:
I was wondering whether the CM API currently (I'm currently using svn
3470) supports establishing UC connections? I have the RC transport type
working fine using the CM.
Unless there's a bug, nothing in the CM should prevent UC from working.
I had a quick look in the
Richard Frank wrote:
Oracle currently depends on 64 bytes of private data for connect and
accept.
Is any of that data used to exchange address information?
It's impossible to provide both the source and destination address in the CM REQ
private data and still give the user 64 bytes. The
Roland Dreier wrote:
Does the patch below help? It looks like there is a missing break
after the UC case in the kernel CM, so the code falls through and
overwrites the field with the value for RC.
Good catch. I overlooked this about 10 times now... The break should be there.
- Sean
Kanevsky, Arkady wrote:
I will update the proposal for IBTA based on this feedback
and all other feedback posted.
I will still separate private data usage proposal
and port mapping one.
Again, I think that these should be in the same proposal. The CM REQ carries
the IB transport layer
Caitlin Bestler wrote:
However, the daemon typically listens on *all* addresses that
the system supports. It is not uncommon for the application
to note which destination address was actually requested and
to vary the service provided based upon that. This is what makes
it possible for single
Pradeep Satyanarayana wrote:
Is there a ballpark estimate (or a plan) of when CMA willl be ready?
Estimates like by end of Q4 2005
or end of Q1 2006 will help us make some decisions if we should submit a
patch for this bug or wait
for CMA.
The kernel CMA is ready today. An additional change
Michael Krause wrote:
This is really an IBTA issue to resolve and to insure that backward
compatibility with existing applications is maintained. Hence, this
exercise of who is broken or not is inherently flawed in that one cannot
comprehend all implementations that may exist. Therefore, the
Will the CMA have that the same function ib_at_route_by_ip() that we are
using in libibat?
The CMA will resolve IB routes based on source/destination TCP/IP addresses if
that is what you are looking for. It will then establish connections based on
those routes. You may want to look at
James Lentini wrote:
Standardizing the protocol will ensure interroperability.
Agreed - just didn't know if this was the responsibility of the SWG.
sean version(8) | reserved(8) | src port (16)
version(1) | reserved(1) | src port (2)
sean src ip (16)
sean dst ip (16)
sean user
Sean Hefty wrote:
version(8) | reserved(8) | src port (16)
src ip (16)
dst ip (16)
user private data (56)/* for version 1 */
Random thought... if the src and dst IP addresses will always be on the same
network, the data could be layed out as:
network addr (x)
src host addr (y)
dst
Tom Tucker wrote:
I'm thinking that for iWARP, there won't be anything in the Private Data
at all except consumer private data. Is that your expectation?
I believe so. This is only trying to define a TCP/IP connection service over
IB. I'm assuming that there's no need to define something
permit UC connections over the CM. I was able to
test this using cmpost.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: cm.c
===
--- cm.c(revision 3830)
+++ cm.c(working copy)
@@ -135,6 +135,7 @@
__be64
Is there a way for a userspace application to know if a device has been added or
removed?
I'm porting the CMA to userspace. One of the features of the kernel CMA is that
listen requests can span all devices, with device add/remove handled
automatically. I'd like to expose similar
Roland Dreier wrote:
Sean Is there a way for a userspace application to know if a
Sean device has been added or removed?
We don't really handle this right now. It could probably be made to
work on top of hotplug/udev/hal/something but it seems tricky to me.
At this point, I'm still
Tom Tucker wrote:
I'm not sure of all the issues you're considering, but it seems to me at
first blush that the user space stuff should talk the kernel CMA. If you
don't do it in the kernel:
- you will end up replicating transport dependent connection management
logic in the user mode library
Sean Hefty wrote:
The following patch should fix the MAD layer's DMA mapping issue. This
patch includes all related patches that were previously posted. The fix
involved changing the MAD layer API. All callers must now use the MAD
layer to allocate and free send MADs. DMA mappings are done
Caitlin Bestler wrote:
- The kernel CMA will be modified to remove the requirement
to use rdma_create_qp(). Users that want to allocate and
manage their own QP states will be able to specify QP
attributes (qpn, qp_type, srq) through the rdma_conn_param structure.
Why?
If the userspace CMA
Caitlin Bestler wrote:
I believe it requires a CM protocol version change, or a IP Address
Header present bit.
Basically, userspace consumers can supply *any* 72 bytes of private data
currently.
To maintain backwards compatability you need an authenticator that says
this IP
header data
Title: Message
Dear OpenIB, SWG and DAT members,
enclosed is teh second version of the proposal.
There are really 2 proposals that are related.
The first one is encoding IP 5-tuple into REQ private data
with small additional info for versioning and IB
capabilities.
Kanevsky, Arkady wrote:
Correct.
But this does bring the question how responder CM knows that it need to
parse
the private data. I suspect this will be done via new version of CM.
But a suage of some of the CM REQ reserved fields are also possible.
Anotherwords the current CM version assumes
Caitlin Bestler wrote:
Is that because you do not agree that there is a problem?
Or is it that you think the gap betweeen this and existing IP
connection semantics is small enough that it is better to cover
it with a disclosure than by changing the CM protocol?
I would define the problem as:
Kanevsky, Arkady wrote:
Think of a single API that supports iWARP and IB (transport independent
API).
The CMA implements this today and did not require any changes to the IB CM.
To a connection listener it provides the IP 5-tuple + private data.
For IB it means that CM parses REQ and
Kanevsky, Arkady wrote:
Sean,
The reason IBTA is interested to address IP address issue
is because of multiple UPLs and APIs want to support
socket based connection model. Sure each one of them
can define its own protocol (for private data).
But this will not ensure interoperability.
There's
Kanevsky, Arkady wrote:
It is APIs not ULPs that are concern.
Yes - and an application that wants to use IP addressing instead of IB
addressing should use a different API than that of the IB CM. Trying to define
the IB CM to use anybody's favorite transport/network address is the wrong
Caitlin Bestler wrote:
What you are proposing is an API that purports to have the
semantics of TCP/IP connection establishment that can be
implemented under non-IP transports such as InfiniBand.
However, as proposed the mapping of this API to InfiniBand
does *not* implement the semantics of
Kanevsky, Arkady wrote:
Sean,
answers in-line.
Arkady
At this point, I'm just going to disagree with this approach and move on with
the current implementation of the CMA. What's needed is a service that provides
IB connections using TCP/IP addressing. I don't believe this proposal meets
Roland Dreier wrote:
This changes alloc_response_msg() in mad_rmpp.c to return the struct
it allocates directly (or an error code a la ERR_PTR), rather than
returning a status and passing the struct back in a pointer param.
This simplifies the code and gets rid of warnings like
Kanevsky, Arkady wrote:
What are you trying to achieve?
I'm trying to define a connection *service* for Infiniband that uses TCP/IP
addresses as its user interface. That service will have its own protocol, in
much the same way that SDP, SRP, etc. do today.
I am trying to define an IB REQ
Sean Hefty wrote:
- The kernel CMA will expose a new call, rdma_init_qp_attr() to
initialize QP attributes used to modify the state of the QP. The call
will be similar to the infiniband CM routine. Use of this call is
optional. The CMA will automatically transition QPs created
Michael S. Tsirkin wrote:
Sounds like a lot of work :).
I think that it's less work than the alternative.
Are there benefits to this approach as opposed to implementing everything
in a library on top of ucm/uverbs?
I considered, and continue to consider implementing on top of ucm. The
Tom Tucker wrote:
FYI, I've started writing the iw_cm that sits below the rdma_cm. Here's
the general picture I have in mind.
+-+
| RDMA CM |
+-+-+-+
| |
++ ++
| |
+-+ +++
| IB CM | |
Michael S. Tsirkin wrote:
Quoting Sean Hefty [EMAIL PROTECTED]:
I considered, and continue to consider implementing on top of ucm. The
drawbacks are: it requires more kernel modules: one for the CM, one for SA
query, and one for address translation.
Cant address translation be done
There's a trade-off between service ID space used by the CMA and the amount of
private data available to the user. Currently, the CMA reserves 64k of service
ID space and provides 56 bytes of user private data. We can give the user 60
bytes of private data space by shifting 3 bytes (plus 1
Roland Dreier wrote:
Sean This results in the CMA reserving 2^40 IDs (about 6% of the
Sean total range).
Just to be nitpicky -- 2^40 service IDs is 1 / 2^24 of the total
number of service IDs (2^64), which is about .06%.
Uhm.. good catch. That makes a difference in how important
Caitlin Bestler wrote:
How much logic is really in the RDMA CM?
If it is sufficiently small, which is what my expectation is,
would it make sense to simply make the IB CM and IW CM conform
to the same polymorphic interface? (Making the RDMA CM little
more than a re-directing inline function).
Caitlin Bestler wrote:
So it sounds like the justification for the RDMA CM being
a distinct module is to centralize handling of device addition
and removal. Beyond that you are incorporating IB-specific
but device-independent logic. As a goal, the iWARP side
should be migrating there as well.
Tom Tucker wrote:
Something that didn't make sense for the kernel rdma_cm running over IB was
adding a backlog parameter to the listen request. (The IB CM is callback
driven, so there's not really a backlog.) I will probably add this to the
userspace API. Does iWarp need a backlog parameter
Roland Dreier wrote:
Sean Is there a way for a userspace application to know if a
Sean device has been added or removed?
We don't really handle this right now. It could probably be made to
work on top of hotplug/udev/hal/something but it seems tricky to me.
Any idea on how we
Michael S. Tsirkin wrote:
But I mean, we can already send ARP packets from userspace, cant we?
That I don't know. Are there APIs to send requests and view responses in
userspace?
- Sean
___
openib-general mailing list
openib-general@openib.org
Here's a modified version of Roland's original patch that adds only
the node_guid to struct ib_device.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
I'll rework my other patches based on this change.
Index: include/rdma/ib_verbs.h
Roland Dreier wrote:
Sean, looks like your MAD send buf stuff may have broken send
timeouts. Any quick ideas before I dig into this?
No quick ideas why. I'll start looking into this as well.
- Sean
___
openib-general mailing list
Roland Dreier wrote:
Sean, looks like your MAD send buf stuff may have broken send
timeouts. Any quick ideas before I dig into this?
I think that the send_handler in user_mad.c is broken.
- Sean
___
openib-general mailing list
Roland Dreier wrote:
Sean I think that the send_handler in user_mad.c is broken.
I don't see anything obviously wrong -- in Jay's log, the call to
ib_free_send_mad() is crashing. When can it be wrong to do that from
the send handler?
I don't see anything off there either. Timeouts seem
Sean Hefty wrote:
I don't see anything off there either. Timeouts seem to work fine with
CM testing, so I'm guessing that the issue is somewhere in user_mad.c.
I'm trying to see if there's anything wrong in ib_umad_write() that
might cause it to crash on the completion.
Re-testing
Roland Dreier wrote:
Good catch. Seems like the below patch is the right fix:
we start out with
Fix looks right to me.
packet-length = length;
I don't think that this assignment is needed. Once the packet is sent, it is
simply freed.
- Sean
some
additional code cleanup.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: user_mad.c
===
--- user_mad.c (revision 3861)
+++ user_mad.c (working copy)
@@ -99,7 +99,6 @@
struct ib_mad_send_buf *msg;
struct
Aniruddha Bohra wrote:
Oh well, I guess this is a different bug. Is there an oops or
anything in your kernel log, or is this just a userspace crash?
This is what I see :
Oct 27 22:03:34 hora-3 OpenSM[7995]: OpenSM Rev:openib-1.1.0
Oct 27 22:03:34 hora-3 kernel: ib_mad: Method 1 already in
probably want to remove node_guid from the device attributes as well.
Signed-off-by: Sean Hefty [EMAIL PROTECTED]
Index: sysfs.c
===
--- sysfs.c (revision 3892)
+++ sysfs.c (working copy)
@@ -622,21 +622,15 @@
static ssize_t
Jay Higley wrote:
I updated to version 3891 and tried it with the 2.6.13.4 Kernel that I
was using and got unresolved symbol errors for kzalloc. I upgraded the
kernel to 2.6.14 and tried agin and got the below compile errors. As an
aside, when I was running the unpatched openSM on a
Kevin Reilly wrote:
Thanks Sean,
I think the rdma_resolve_addr() does what we want. Translate a local IP
to a ib_device structure that i can use in the ibverbs.
What we want to do is pretty simple and we won't need to create a
connection.
Based on your description, I think that
Jay Higley wrote:
I compiled the openIB stack with the 2.6.14 kernel and three modules
(ib_sdp, ib_at, and ib_addr) will not load due to missing symbol
ip_dev_find. I see the source for this routine in kernel/net, but
apparently it didn't get compiled into my kernel. Does anyone know what
Sean, Hal, now that 2.6.14 is out, do you plan to apply
the patches in https://openib.org/svn/gen2/trunk/src/linux-kernel/patches/?
Once you do, I'll put reverted patches in the backport directory.
I'll apply the patch to addr.c shortly. Thanks for the reminder.
- Sean
I'm using the new rdma cma interface and i've perhaps stumbled onto a bug. I'm
trying to bind to port on both IB ports of a
mthca device. The IPoIB interfaces for the HCA are configured as two seperate
subnets. The second rdma_listen() always fails with
How are you binding the address to
Fix for bug in cma_ib_listen().
Set cm_id to NULL after destroying the listen ib_cm_id so rdma_destory_id()
doesn't try and destroy it again later.
Thanks - applied.
- Sean
___
openib-general mailing list
openib-general@openib.org
Roland Dreier wrote:
Something like this is probably required for ucm and anything else
that exports a character device, since everyone seems to have copied
my bad user_mad code. But I haven't had a chance to do anything
beyond user_mad and uverbs so far...
Thanks for the info. I'll take a
yipee wrote:
I think that I've noticed a problem in compiling user applications with a
different compiler than the running-kernel modules compiler (x86 32bit vs.
64bit). For compiling an openib application on a 32bit x86 and running it on a
64bit AMD Opteron.
When compiling a program with a
Roland Dreier wrote:
I think the real fix is just to fix the declaration so that the
structure is laid out the same for all architectures, and bump the ABI
version yet again.
I think that we'll need a similar update to cm_abi.h too.
- Sean
___
yipee wrote:
I think that I've noticed a problem in compiling user applications with a
different compiler than the running-kernel modules compiler (x86 32bit vs.
64bit). For compiling an openib application on a 32bit x86 and running it on a
64bit AMD Opteron.
I've checked in Roland's patch,
Roland Dreier wrote:
All structs more than 4 bytes in size have to be padded to a multiple
of 8 bytes, or else they end up with a different size between 32-bit
and 64-bit architectures. I think something like the patch below
along with the corresponding userspace libibcm change is required.
Roland Dreier wrote:
I just committed the following patch for user_mad.c, which fixes
various issues with possibly freeing various data structures before
the last reference is gone. For example, cdev_del() might return
before the last reference to the cdev is gone, so freeing a structure
Roland Dreier wrote:
Sean I can't say that I grasp the relationship between the cdev_*
Sean and class_* calls yet, but should umad and ucm create their
Sean own classes? I'm trying to add the ucma, and I'm wondering
Sean if we should add another infiniband_blah class, versus
Roland Dreier wrote:
Sean No - it's not attached to a device. I was going to create
Sean just one character device, which is why I was wondering if
Sean creating a new class was the right approach.
It might just make sense to use the existing misc class I guess.
It appears that
Roland Dreier wrote:
Anyone have any objection to me committing the following patch?
I have no objection.
- Sean
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please
I'm implementing the userspace CMA and noticed that there are a couple of areas
where userspace support overlaps.
For example, both the CMA and IB CM need to copy path records between userspace
and the kernel. They also copy QP attributes, which would also be needed by
verbs at some point to
Michael S. Tsirkin wrote:
Common header files/structures might make some sense, but what would
the routines do, besides copy to/from user?
Could you give an example?
The copies aren't memory copies, but field by field assignments. The function
below is used by ib_ucm to copy QP attributes
yipee wrote:
Is there some way to view the list of current CM end points in their various
states (listen,connection)?
Nothing like this is available today. I can record this as something to add in
the future, but it's unlikely to be a high priority for at least a few weeks.
- Sean
The following patch would expose support in uverbs to define a QP attribute
structure that could be used by other kernel modules (e.g. the IB CM and CMA)
needing to exchange QP attribute information with a userspace library.
I've only compile tested this patch, since I'm only looking for feedback
Roland Dreier wrote:
Seems OK but maybe we should create a new file (uverbs_marshall.c?)
Sounds fine, but see below...
I very carefully made ib_user_verbs.h a file that did not include any
kernel internals and could be safely included from userspace. So this
needs to go in a different file
yipee wrote:
Maybe I can write some initial implementation.
If you could do that, it'd be great. I think something like this would be
useful. I just don't know if I'll have time to add it soon.
- Sean
___
openib-general mailing list
Grant Grundler wrote:
But as Roland said, RDS doesn't even have a Makefile.
I've reviewed some of it shortly after it got dropped in but
still need to go through alot more of the code.
The code that I've looked at doesn't appear to be written to any of the openib
code. A Makefile wouldn't
Tom Tucker wrote:
Sean:
I was looking through ip_resolve_local and it looks to me like
if the source address is 0, it will end up getting set to the
destination IP instead of the IP address of the local interface.
The intent of ip_resolve_local() is to check if a given destination address is
Rick Frank wrote:
No we do not use TCP sockets - we use to many connections for this 100k+.
Isn't RDS implemented on top of reliable IB/RDMA connections anyway?
- Sean
___
openib-general mailing list
openib-general@openib.org
Steve Wise wrote:
The intent of ip_resolve_local() is to check if a given destination
address is on the local system. If it is and no source address is
specified, then the source address is set to the same address as the
destination.
This doesn't sound correct to me. The src ip address is
Tom Tucker wrote:
Well It won't necessarily fail will it? If you specified the source
address as another port on the same machine, but NOT the one with
connectivity to the remote peer, the routine will succeed, but the
results are not what you expect...and you will fail further down the
line
Tom Tucker wrote:
If you specify a 0 as a source address, won't the private data contain
the destination address as the source address, or did I miss something?
The source and destination addresses will be the same if the destination is on
the local system.
E.g. The local system has address
Is there a way to get the node_type for an ibv_device?
- Sean
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Any comments on changing the signature of the struct ib_device
resize_cq method to take the new CQ size rather than a pointer to the
new CQ size? The low-level driver would then be responsible for
updating the cq-cqe member itself (possibly with proper locking).
This would also make the prototype
701 - 800 of 2277 matches
Mail list logo
