One advantage of having shared objects is to be able to preserve IB connections across process restarts. If the traffic is not very high and the buffers are in shared memory (which I think should be), then it can save connection setup and message recovery time.
Shouldn't the protocol to create and destroy and pass the various IB objects around be decided by the specific application rather than the library trying to solve this problem? Thanks Ganesh On 6/26/07, Roland Dreier <[EMAIL PROTECTED]> wrote:
> This is not directly related to SRC: this is an effort > to make it possible to share QPs, CQ etc across processes > in the same way as they can be currently shared across threads. > So assuming that we want multiple processes to post to > the same QP, how can we support this? This looks like a lot of work for an unknown gain. Who is going to really use this? ie is it worth the trouble? > > - Given that everything shared is in shared memory, > > I think we should try and keep shared memory usage to minimum. > For example, in mthca mr object just needs a key: we could > keep it in non-shared memory, just pass the key around > and save on sahred memory usage. This comment made me realize there are a few more problems here. What happens if I do ibv_reg_mr() in one process, pass the MR to another process, and then do ibv_dereg_mr() in the second process? What about if someone registers a region in shared memory -- are there any fork/copy-on-write issues with that? I think there are probably bugs in the locked_vm accounting in the kernel right now -- it doesn't take into account the possibility of passing context fds from one process to another. In general what do you think the rules for destroying objects should be? What if process A creates a QP, passes it to process B, and then process A dies? Should the QP still be usable? Should process B be able to destroy it? What if process A is still alive -- should process B be able to destroy the QP? > We need to share file descriptors too. Is there a way to pass these > around besides unix domain sockets? I guess we need this to be able to re-mmap doorbell pages etc, right? I wonder if there's a better way around that... maybe extending the kernel interface so that unrelated processes can share a context, eg by putting contexts in a filesystem or something like that. > But are you sure we want to break API for all users just to add > a new capability for a minority that wants shared memory support? Yes, you're right... better to be backward compatible and have a new API for shared stuff. - R. _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
_______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
