No problem with me. Why don't we then modify this RFC to simply eliminate the --enable-opal-progress-thread option, and leave the rest as-is so we can enable the opal thread machinery without enabling MPI thread multiple?
I agree that having an ORTE-level thread makes the most sense, if it is needed. On Mar 10, 2010, at 10:20 AM, George Bosilca wrote: > There was way too much information on this thread that I was looking for ;) > And a lot of misunderstandings ... > > If we want to allow ORTE to be on his own thread, then we should clearly > banish the progress_thread from this equation. I would prefer ORTE to be as > separated from the rest of the MPI library as possible, and therefore avoid > most of the locks and overheads on the MPI itself. Moving ORTE (as it only > use TCP sockets) on it's own poll is the best looking approach, and this can > be easily done once we upgrade out libevent to the 2.0. > > To be honest the progress thread was not the smartest idea we had. It makes > the code more complex, and the added benefit is quite small. Again, once we > move to the libevent-2 we will cleanup the code, and have a more consistent > approach. > > george. > > On Mar 8, 2010, at 10:11 , Ralph Castain wrote: > >> >> On Mar 8, 2010, at 6:43 AM, Jeff Squyres wrote: >> >>> On Mar 7, 2010, at 8:13 PM, Ralph Castain wrote: >>> >>>>> How about calling it --enable-opal-event-progress-thread, or even >>>>> --enable-open-libevent-progress-thread? >>>> >>>> Why not add another 100+ characters to the name while we are at it? :-/ >>>> >>> >>> :-) >>> >>> I didn't really think the length mattered here, since it's a configure >>> argument. There has been a *lot* of confusion over the name of this >>> particular switch over the past few years, so I'm suggesting that a longer, >>> more descriptive name might be a little better. Just my $0.02... >> >> I honestly don't think that is the source of the confusion. The revised name >> tells you exactly what that configure option does - it enables a thread at >> the opal layer that calls opal_progress. Period. >> >> The confusion is over how that is used within the code, given that opal >> doesn't have any communication system (as George pointed out). So having an >> opal progress thread running will cause the event library to tick over, but >> that does....? It isn't directly tied to any existing subsystem, but rather >> cuts across any of them that are sitting on sockets/file descriptors etc. >> using the event library. >> >> If you look at the other progress threads in the system (e.g., openib), >> you'll find that they don't use the event library to monitor their fd's - >> they poll them directly. So enabling the opal progress thread doesn't >> directly affect them. >> >> So I would say let's leave the name alone, and change it if/when someone >> figures out how to utilize that capability. >> >>> >>>>> The openib BTL can have up to 2 progress threads (!) -- the async verbs >>>>> event notifier and the RDMA CM agent. They really should be >>>>> consolidated. If there's infrastructure to consolidate them via opal or >>>>> something else, then so much the better... >>>> >>>> Agreed, though I think that is best done as a separate effort from this >>>> RFC. >>>> >>> >>> Agreed -- sorry, I wasn't clear. I wasn't trying to propose that that work >>> be added to this RFC; I was just trying to mention that there could be a >>> good use for the work from this RFC if such infrastructure was provided. >>> >>>> I believe there was a concern over latency if all the BTLs are driven by >>>> one progress thread that sequentially runs across their respective file >>>> descriptors, but I may be remembering it incorrectly... >>>> >>> >>> >>> FWIW, I believe the openib progress threads were written the they way they >>> were (i.e., without any opal progress thread support) because, at least in >>> the current setup, to get the opal progress thread support, you have to >>> turn on all the heavyweight locks, etc. These two progress threads now >>> simply pthread_create() and do minimal locking between the main thread and >>> themselves, without affecting the rest of the locking machinery in the code >>> base. >>> >>> I'm not saying that's perfect (or even good); I'm just saying that that's >>> the way it currently is. Indeed, at a minimum, Pasha and I have long >>> talked about merging these two progress threads into 1. It would be even >>> better if we could merge these two project threads into some other >>> infrastructure. But it's always been somewhat of a low priority; we've >>> never gotten a round tuit... >>> >>> -- >>> Jeff Squyres >>> jsquy...@cisco.com >>> For corporate legal information go to: >>> http://www.cisco.com/web/about/doing_business/legal/cri/ >>> >>> _______________________________________________ >>> devel mailing list >>> de...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/devel >> >> >> _______________________________________________ >> devel mailing list >> de...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/devel > > > _______________________________________________ > devel mailing list > de...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/devel
