> Generally, I think it would be nice if this could work > in the same way as with multiple threads: a single process does > destroy, the rest must not use the same object after this, > synchronisation it up to the app. > > But you made me realise that we need an API for non-controlling processes to > release the userspace resources without destroying the kernel-level object.
What is a non-controlling process? To the uverbs code in the kernel there is only one file structure that happens to be shared by multiple processes. But they are all equal. > > I think there are probably bugs > > in the locked_vm accounting in the kernel right now -- it doesn't take > > into account the possibility of passing context fds from one process > > to another. > > Hmm, might be a good idea to fix the bugs anyway, no? Yes, I guess we need to take a reference on the mm structure in ib_umem_get() and only drop it after we free the umem. > > Should process B be > > able to destroy it? What if process A is still alive -- should > > process B be able to destroy the QP? > > I think in practice a single process will do this. > My approach generally is: let's have same rules as for multiple threads. I don't think it's quite as simple as saying that it's just like multiple threads. Creating/destroying QPs from a PD shared by multiple processes opens lots of problems. Let's take the mthca case: the userspace driver needs to have a table of QPN -> QP struct so that it can look up which QP a completion belongs to. This means that if process A and process B share QP X, then X has to be in the QP table of both processes. OK, that's fine, when QP X gets passed from A to B, then B can put it in the table. But what happens when B destroys QP X? How does process A know to take X out of its table? What if process A has died in the meantime? Or what if process A and process B share CQ 1, and process B creates QP Y in a non-shared PD but attaches it to CQ 1? What happens when process A polls a completion for QP Y from CQ 1? > > I guess we need this to be able to re-mmap doorbell pages etc, right? > > I wonder if there's a better way around that... maybe extending the > > kernel interface so that unrelated processes can share a context, eg > > by putting contexts in a filesystem or something like that. > > Hmm, I don't have principal objection, however this would mean > we'd have to change kernel-user interface again. the proposed > API extensions can mostly be done in userspace only. > > And it seems to me like much more work that just let the app > use unix domain sockets, for me. What are the advantages of this approach? > > Further, since there is already an existing kernel interface for this, > should we be inventing our own? The advantage is that sharing objects in a filesystem by doing open(), protected by permissions etc. is much more familiar than passing fds through sockets. I'm not sure it makes sense but shared memory + unix domain socket fd passing is not a very natural way for most people to program. - R. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
