Am 16.10.2012 16:25, schrieb Stefan (metze) Metzmacher: > @Yann and Steve: thanks for the feedback! > >>>> I'm currently researching how to implement SMBDirect [MS-SMBD] >>>> together with the multi channel feature of SMB 3.0 in Samba. >>>> >>>> As Samba currently uses one process per tcp connection >>>> and maintains a lot of in memory state within the process >>>> (e.g. for the SMB_VFS modules) it would require a lot of work >>>> to change Samba to coordinate two (or more) processes for one logical >>>> multi channel connection. >>>> >>>> My current plan tries to pass the socket fd of new connections >>>> (which join an existing multi channel session) via fd-passing to >>>> the existing process. >>>> >>>> Now I'm wondering if this would also be possible with >>>> a rdma connection (struct rdma_cm_i ). >>>> >>> RDMA / verbs ressources are tied to a process (especially Memory >>> Registration), but it's ending up in the HCA, which is probably unaware >>> of processes. >>> >>> Additionally, an RDMA_CM connection is not identified by a FD, so this >>> kind of Unix trick (FD passing through Unix socket: SCM_RIGHTS) is not >>> going to work. >>> >>> Forking might already be a challenge for a RDMA/verbs application, so I >>> don't think that sharing/moving an RDMA_CM connection across different >>> processes is supported. >>> >>> But other people on this list (especially Roland Dreier and Sean Hefty) >>> could find a solution. >>> >>> Regards. >>> >> >> fork() support like you need is not there in Linux RDMA verbs. Another >> alternative is to fork() before you setup the RDMA connection. IE if a >> regular TCP socket is first used to negotiate RDMA mode, then maybe you >> could fork() after negotiation but before setting up the RDMA connection >> and other resources? > > For client connections that would work, but it can't work for a server > that uses fork() after listen(). > > Would it be possible to extend the librdmacm and libibverbs to: > - handle fork() after getting RDMA_CM_EVENT_CONNECT_REQUEST > which means there have to be some functions to destroy contexts like > struct rdma_cm_id, but without affecting the real connection. > Similar to close() on an FD that is used shared between two processes. > - export/import an interprocess token attached to the hierarchy of objects > similar to gss_export_sec_context/gss_import_sec_context. > Or a function which exports the contexts into a FD (of a pipe), > which could also transfer open FDs to the other end. > Maybe this could pass the information through a rdma/ibverb event channel?
It would be ok if there's a limitation, which means that the transfer to a different process is only possible if there're: - no posted work requests - no queue pairs - completion queues active on the connection. metze
signature.asc
Description: OpenPGP digital signature
