> Quoting Gleb Natapov <[EMAIL PROTECTED]>: > Subject: Re: Re: [PATCH RFC] sharing userspace IB objects > > On Tue, Jun 26, 2007 at 12:51:25PM +0300, Michael S. Tsirkin wrote: > > > Quoting Gleb Natapov <[EMAIL PROTECTED]>: > > > Subject: Re: Re: [PATCH RFC] sharing userspace IB objects > > > > > > On Tue, Jun 26, 2007 at 10:06:41AM +0300, Michael S. Tsirkin wrote: > > > > > Is there really a strong use case for making every type of object > > > > > shareable? Can we handle the SRC stuff without going to this > > > > > extreme of complexity? > > > > > > > > This is not directly related to SRC: this is an effort > > > > to make it possible to share QPs, CQ etc across processes > > > > in the same way as they can be currently shared across threads. > > > > So assuming that we want multiple processes to post to > > > > the same QP, how can we support this? > > > > > > Are you absolutely sure you even want to support this? > > > > Take a look here :) > > http://www.quotedb.com/quotes/1007 > So there is still a chance you'll reconsider :)
Sure, if someone comes up with a better way to improve scalability for single-threaded applications. > > > > > What is the user case? > > > > Use case? Scalability. Pls go over Dror's presentation given at Sonoma - > > he calls this SSQ. > > As far as I can tell he is talking about HW supported solution and not > half baked SW one. No, sharing a send queue must be done in software. I don't really see the reason for sarcasm: do you see value in sharing resources between multiple threads? Why not multiple processes? Some people just don't want to program in multithreaded environment. > > > > > If multiple processes what to post to the same QP how will you > > > ensure that right process will receive right completion event? > > > > Same as with threads - memory for CQEs and locks will be allocated > > in shared memory to make it possible for multiple processes to poll > > CQ simultaneously, and they get completions in FCFS order. > > What to do with them is up to the user. > > Are you going to use this API? How? There is no point in discussing user > API without specifying HOW user will be using it. You have to ask what > user want and design your API accordingly and not other way around. > So suppose I want to use proposed API to implement super scalable MPI. We'd come up with MPI_Send implementation inside libibverbs:). Think layered - I'd like to make a minimal possible API change to make scalability improvements possible. > I setup shared QP/CQ/... and each rank start to post into the QP and > receive completion from CQ and suppose rank A picked completion that > belongs to rank B so I will need to setup out of band channel to pass > this completion from A to B. This is not looks good at all to me. This is not different from multiple threads sharing a CQ, really - and we do support this already. In the part of the message that you have cut out, I showed some use cases that avoid this "side channel" (which could be just shared memory btw). -- MST _______________________________________________ general mailing list [email protected] http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
