Michael S. Tsirkin wrote:
So, since the need seems to be there, I started thinking about how this could
be done.
Basically, we could create shared memory objects (shm_open) and use these
for all hardware-accessible registers, as well as necessary control (head/tail
pointers,
spinlocks used for protection, etc).
If we do this, we can use unix domain sockets for everything,
a client just mmaps the fd that it got. Does this make sense?
OK, here's a draft showing how an API to do this could look like.
Basically the idea is that we'd ask low-level drivers to provide an
(optional) API to
1. allocate context and all its objects inside a shared memory object
2. pack and unpack objects from/to unix domain socket messages
So to share a QP, the server would
A. open shared context, create pd, cq, qp
B. listen on unix domain socket
C. pack the context, pd, cq, qp
D. send them to clients that connect
The client would
A. create unix domain socket
B. connect to server
C. get message from server
D. unpack context, pd, cq, qp
One problem here (which annoys for long time...) is that typically the
active side of a connection is the one that sends the first packet and
hence you must post receives to the QP --before-- accepting the
connection request.
So, if both sides use a shared-context, they would need to implement a
synchronization protocol (that is don't deliver established event to the
active before the passive accepted).
And, if the active side does not use shared context where the passive
side does use shared context, you need either the shared context to
allocate/post receives from shared-memory or rely on RNR NAKs, what do
you think?
Also, what was your thinking on registering the QP/CQ memory? is the
plan to implement a verb for registering shared-memory as was in the
VAPI stack, or you want to register this memory as "just" virtual?
Or.
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
