@Yann and Steve: thanks for the feedback! >>> I'm currently researching how to implement SMBDirect [MS-SMBD] >>> together with the multi channel feature of SMB 3.0 in Samba. >>> >>> As Samba currently uses one process per tcp connection >>> and maintains a lot of in memory state within the process >>> (e.g. for the SMB_VFS modules) it would require a lot of work >>> to change Samba to coordinate two (or more) processes for one logical >>> multi channel connection. >>> >>> My current plan tries to pass the socket fd of new connections >>> (which join an existing multi channel session) via fd-passing to >>> the existing process. >>> >>> Now I'm wondering if this would also be possible with >>> a rdma connection (struct rdma_cm_i ). >>> >> RDMA / verbs ressources are tied to a process (especially Memory >> Registration), but it's ending up in the HCA, which is probably unaware >> of processes. >> >> Additionally, an RDMA_CM connection is not identified by a FD, so this >> kind of Unix trick (FD passing through Unix socket: SCM_RIGHTS) is not >> going to work. >> >> Forking might already be a challenge for a RDMA/verbs application, so I >> don't think that sharing/moving an RDMA_CM connection across different >> processes is supported. >> >> But other people on this list (especially Roland Dreier and Sean Hefty) >> could find a solution. >> >> Regards. >> > > fork() support like you need is not there in Linux RDMA verbs. Another > alternative is to fork() before you setup the RDMA connection. IE if a > regular TCP socket is first used to negotiate RDMA mode, then maybe you > could fork() after negotiation but before setting up the RDMA connection > and other resources?
For client connections that would work, but it can't work for a server that uses fork() after listen(). Would it be possible to extend the librdmacm and libibverbs to: - handle fork() after getting RDMA_CM_EVENT_CONNECT_REQUEST which means there have to be some functions to destroy contexts like struct rdma_cm_id, but without affecting the real connection. Similar to close() on an FD that is used shared between two processes. - export/import an interprocess token attached to the hierarchy of objects similar to gss_export_sec_context/gss_import_sec_context. Or a function which exports the contexts into a FD (of a pipe), which could also transfer open FDs to the other end. Maybe this could pass the information through a rdma/ibverb event channel? Are there any other possible solutions? metze
signature.asc
Description: OpenPGP digital signature
