Hi Chris Thanks for the patience and the clarification - much appreciated. In fact, I have someone that needs to learn more about the code base, so I think I will assign this to him. At the least, he will have to learn a lot more about the mapper!
I have no problem with modifying the pernode behavior to deal with the case of someone specifying -np as you describe. It would be relatively easy to check. As I understand it, you want the behavior to be: 1. if no -np is specified, launch one proc/node across entire allocation 2. if -np n is specified AND n is less than the number of allocated nodes, then launch one proc/node up to the specified number. Of course, this is identical to just doing -np n -bynode, but that's immaterial. ;-) 3. if -np n is specified AND n is greater than the number of allocated nodes, error message and exit 4. add a -npernode n option that launches n procs/node, subject to the same tests above. Can you confirm? Finally, I think you misunderstood my comment about the MPI folks. Our non-MPI users couldn't care less about commonality of command line arguments across MPI implementations. Hence, I leave issues in that area to the MPI members of the team - they are the ones that decide how to deal with the myriad of different option syntaxes in the MPI community. Gives me too much of a headache! :-) Ralph On 11/28/06 10:58 PM, "Maestas, Christopher Daniel" <cdma...@sandia.gov> wrote: > Ralph, > > Thanks for the feedback. Glad we are clearing these things up. :-) > > So here's what osc mpiexec is doing now: > --- > -pernode : allocate only one process per compute node > -npernode <nprocs> : allocate no more than <nprocs> processes per > compute node > --- > >> Cdm> I think I originally requested the -pernode feature. :-) I've >> Cdm> seen >> one issue I know of ... >> When used in the following manner: >> --- >> "mpiexec -pernode -np N" and if N is > allocated nodes, it should >> error out and not proceed. I need to update/learn to update the trac >> ticket on this. >> --- > This is an incorrect usage - the "pernode" option is only intended to be > used without any specification of the number of processes. Pernode > instructs the system to spawn one process/node across the entire > allocation - we simply ignore any attempt to indicate the number of > processes. > > I suppose I could check and error out if you specify the number of procs > AND --pernode. I would have to check the code, to be honest - I may > already be doing so. Just don't remember :-) > > CDM> I think I remember looking through the code here, and thinking that > app->num_procs needed to be compared to the > opal_list_get_size(&master_node_list) given, but didn't dig into how > that got set -np when specified. My intention is that we could do for > an N node job scheduled allocation such as torque: > --- > "mpiexec -pernode myprogram" > "mpiexec -np n -pernode myprogram" where n <= N > --- > That is how I believe osc mpiexec is behaving. I tested on 2 nodes and > saw this: > --- > $ mpiexec -comm=none -pernode hostname > rv272 > rv270 > $ mpiexec -comm=none -pernode -np 1 hostname > rv272 > $ mpiexec -comm=none -pernode -np 3 hostname > mpiexec: Error: constrain_nodes: argument -n specifies 3 processors, but > only 2 available after processing -pernode flag. > --- > This was my original intention of requesting -pernode in the first > place. I apologize if I didn't provide this example as well. :-) > Outside of a job scheduler I would think in that cause you would have > the following to launch commands do the same thing: > > "mpiexec -np n -pernode --machinefile=m myprogram " > "mpiexec -np n --bynode --machinefile=m" > "OMPI_MCA_rmaps_base_schedule_policy=node; mpiexec -np n --machinefile=m > myprogram" > > Here the nodes N is basically "wc -l" of the file m and n <=N still > holds true when using -pernode. It may prove difficult to check if > using -pernode, and check in w/ all orted's to see if they've already > launched the process on their nodes. I think that would have to be done > if using -pernode though. If it's too difficult simply error'ng out > when using -pernode and -np may be an easier choice for now. :-) > > ... > >> Cdm> it was more my hope that OSC mpiexec and Open MPI mpiexec options >> would eventually merge into common options. A guy can dream can't he? >> ;-) > > Guess that's up to the MPI folks on the team - from an RTE perspective, > I don't have an opinion either way. > > CDM> I would think that the mpiexec/orterun feature set could launch > other types of executables that weren't mpi based, so I wouldn't think > it would be just mpi spawning. :-) > For example: > "mpiexec -pernode hostname" > > Thanks, > -cdm > > > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
