On 4 May 2010, at 15:41, Jeff Squyres wrote: > On May 4, 2010, at 9:32 AM, Ashley Pittman wrote: > >>> One thing to be careful with a run-time check is that you might not want >>> *all* processes on a box to try to alloc a sysv segment, fork a child, try >>> to connect, ...etc. With large count boxen, you might run out of sysv >>> shmem segments if all procs try the test and/or run into OS serialization >>> issues (someone here at the Forum cited a 96 core box). So you might want >>> to have local rank 0 (or the orted? ...but that wouldn't work for srun / >>> direct launch, etc.) do a test and communicate the results to the rest of >>> the local procs -- maybe in the modex? >> >> I think as as user I'd be quite surprised if my MPI job was spawning >> sub-processes during MPI_Init(). > > Point noted. But actually -- can you give specific reasons as to why a user > should care? Keep in mind that this would be a short-lived fork'ed process > -- not "spawn" in the MPI sense of the word.
You might be running the job under Valgrind or another debugger, bclr has some issues with fork as I remember and traditionally there have been IB mapping issues here as well. I'm sure you could make a case against any of those points if you wanted to but I think the argument stands, doing this kind of run-time check shouldn't be needed. It might be possible to construct the code however so that if it failed to initialise it just wasn't used rather than aborted the job which would have much the same effect as a run-time test but without having to fork new processes and create short-lived shared memory regions. I should of course said fork where I mentioned spawn above to avoid any confusion, spawn has a specific meaning in the context of MPI. I still think a better understanding of the issue is required before any decision here is made though, I'm surprised by Samuels description of the problem because it's not how I remember it and from what Chris says it doesn't reflect what is in linux Git code either. I'd like to see why there is an apparent difference in behaviour before a decision is made to only support one. Ashley. -- Ashley Pittman, Bath, UK. Padb - A parallel job inspection tool for cluster computing http://padb.pittman.org.uk
