On 12/12/06 9:18 AM, "Maestas, Christopher Daniel" <cdma...@sandia.gov>
wrote:
> Ralph,
>
> I figured I should of run an mpi program ...here's what it does (seems
> to be by-X-slot style):
> ---
> $ /apps/x86_64/system/mpiexec-0.82/bin/mpiexec -npernode 2 mpi_hello
> Hello, I am node an41 with rank 0
> Hello, I am node an41 with rank 1
> Hello, I am node an39 with rank 4
> Hello, I am node an40 with rank 2
> Hello, I am node an38 with rank 6
> Hello, I am node an39 with rank 5
> Hello, I am node an38 with rank 7
> Hello, I am node an40 with rank 3
> ---
Clearly mapping by slot here. Question: our default mapping mode is by slot,
as this example also shows. I could turn the npernode behavior around from
what I previously described - we could default to mapping by slot (as we do
normally, and in concert with what mpiexec appears to be doing), and let
users provide the "-bynode" option to change to mapping by node.
This wouldn't have any affect on the pernode behavior since that option only
maps one proc/node anyway (so mapping by slot or by node yields the same
result).
It seems to me this would be more consistent. Make sense?
Ralph
>
> What you describe makes sense to me. Thanks!
>
>> -----Original Message-----
>> From: users-boun...@open-mpi.org
>> [mailto:users-boun...@open-mpi.org] On Behalf Of Ralph Castain
>> Sent: Monday, December 11, 2006 10:27 PM
>> To: Open MPI Users
>> Subject: Re: [OMPI users] Pernode request
>>
>> Hi Chris
>>
>> I have also implemented --npernode N now as well - it is in
>> the trunk as of
>> r12826.
>>
>> The testing you show below using mpiexec really doesn't tell
>> us the story -
>> we need to know the rank of the various processes (and unfortunately,
>> hostname just tells us the host). There is no way to tell the
>> rank from just
>> the order in which the host names are printed to the screen.
>> I have a test
>> program in our distribution (see
>> orte/test/mpi/hello_hostname.c) that will
>> output both the rank and hostname - it would give us the
>> required info.
>>
>> Regardless, I think it would make sense to provide the flexibility you
>> describe. What if we selected this "by-X-slot" style by using
>> the --npernode
>> option, and allowing the user to combine it with the existing
>> "--byslot"
>> option? This would still launch N procs/node, but with the
>> ranking done by
>> slot. If the user doesn't specify "byslot", then we default
>> to assigning
>> ranks by node.
>>
>> Make sense? If so, I can probably have that going before the holiday.
>>
>> Ralph
>>
>>
>>
>> On 12/11/06 7:51 PM, "Maestas, Christopher Daniel"
>> <cdma...@sandia.gov>
>> wrote:
>>
>>> Hello Ralph,
>>>
>>> This is great news! Thanks for doing this. I will try and
>> get around
>>> to it soon before the holiday break.
>>>
>>> The allocation scheme always seems to get to me. From what
>> you describe
>>> that is how I would have seen it. As I've gotten to know
>> osc mpiexec
>>> through the years I think they like to do a first fit
>> approach, but now
>>> that I test it I think the feature needs more testing or
>> I'm not testing
>>> appropriately :-)
>>> ---
>>> $ /apps/x86_64/system/mpiexec-0.82/bin/mpiexec -comm=none
>> -npernode 2
>>> grep HOSTNAME /etc/sysconfig/network
>>> HOSTNAME="an56"
>>> HOSTNAME="an56"
>>> HOSTNAME="an55"
>>> HOSTNAME="an53"
>>> HOSTNAME="an54"
>>> HOSTNAME="an55"
>>> HOSTNAME="an53"
>>> HOSTNAME="an54"
>>> ---
>>>
>>> I guess I would wonder if it would be possible to switch
>> from the method
>>> what you suggest and also allow a "by-X-slot" style of
>> launch where you
>>> would see for npernode = X and N nodes:
>>> proc1 - node1
>>> proc2 - node1
>>> ...
>>> proc(X*1) - node1
>>> ...
>>> proc(X+1) - node2
>>> proc(X+2) - node2
>>> ...
>>> proc(X*2) - node2
>>> ...
>>> proc(N*X-(X-0)) - nodeN
>>> proc(N*X-(X-1)) - nodeN
>>> ...
>>> proc(X*N-1) - nodeN
>>> proc(X*N) - nodeN
>>>
>>> I think that's how to best describe it. Basically you load
>> until there
>>> are X processes on each node before moving to the next.
>> This may prove
>>> to be more challenging, and I can understand if it would
>> not be deemed
>>> "worthy." :-)
>>>
>>> -cdm
>>>
>>>> -----Original Message-----
>>>> From: users-boun...@open-mpi.org
>>>> [mailto:users-boun...@open-mpi.org] On Behalf Of Ralph Castain
>>>> Sent: Monday, December 11, 2006 5:41 PM
>>>> To: Open MPI Users
>>>> Subject: Re: [OMPI users] Pernode request
>>>>
>>>> Hi Chris
>>>>
>>>> Okay, we have modified the pernode behavior as you requested
>>>> (on the trunk
>>>> as of r12821)- give it a shot and see if that does it. I have
>>>> not yet added
>>>> the npernode option, but hope to get that soon.
>>>>
>>>> I have a question for you about the npernode option. I am
>>>> assuming that you
>>>> want n procs/node, but that you want it mapped by NODE. For
>>>> example, proc 0
>>>> goes on the first node, proc 1 goes on the second node, etc.
>>>> until I get one
>>>> on each node; then I wrap back to the beginning and do this
>>>> again until I
>>>> get the specified number of procs on each node.
>>>>
>>>> Correct?
>>>> Ralph
>>>>
>>>>> Ralph,
>>>>
>>>>> I agree with what you stated in points 1-4. That is what we
>>>> are looking
>>>>> for.
>>>>> I understand your point now about the non-MPI users too. :-)
>>>>>
>>>>> Thanks,
>>>>> -cdm
>>>>
>>>>>> -----Original Message-----
>>>>>> From: users-bounces_at_[hidden]
>>>> [mailto:users-bounces_at_[hidden]] On
>>>>>> Behalf Of Ralph Castain
>>>>>> Sent: Wednesday, November 29, 2006 8:01 AM
>>>>>> To: Open MPI Users
>>>>>> Subject: Re: [OMPI users] Pernode request
>>>>>>
>>>>>> Hi Chris
>>>>>>
>>>>>> Thanks for the patience and the clarification - much
>> appreciated. In
>>>>>> fact, I have someone that needs to learn more about the
>>>> code base, so I
>>>>>> think I will assign this to him. At the least, he will have
>>>> to learn a
>>>>>> lot more about the mapper!
>>>>>>
>>>>>> I have no problem with modifying the pernode behavior to
>>>> deal with the
>>>>>> case of someone specifying -np as you describe. It would be
>>>> relatively
>>>>>> easy to check. As I understand it, you want the behavior to be:
>>>>>>
>>>>>> 1. if no -np is specified, launch one proc/node across
>>>> entire allocation
>>>>>>
>>>>>> 2. if -np n is specified AND n is less than the number
>> of allocated
>>>>>> nodes, then launch one proc/node up to the specified
>>>> number. Of course,
>>>>>> this is identical to just doing -np n -bynode, but that's
>>>> immaterial.
>>>>>> ;-)
>>>>>>
>>>>>> 3. if -np n is specified AND n is greater than the number
>>>> of allocated
>>>>>> nodes, error message and exit
>>>>>>
>>>>>> 4. add a -npernode n option that launches n procs/node,
>>>> subject to the
>>>>>> same tests above.
>>>>>>
>>>>>> Can you confirm?
>>>>>>
>>>>>> Finally, I think you misunderstood my comment about the MPI
>>>> folks. Our
>>>>>> non-MPI users couldn't care less about commonality of
>> command line
>>>>>> arguments across MPI implementations. Hence, I leave issues
>>>> in that area
>>>>>> to the MPI members of the team - they are the ones that
>>>> decide how to
>>>>>> deal with the myriad of different option syntaxes in the
>>>> MPI community.
>>>>>>
>>>>>> Gives me too much of a headache! :-)
>>>>>>
>>>>>> Ralph
>>>>
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> us...@open-mpi.org
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> us...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> _______________________________________________
>> users mailing list
>> us...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>
>
> _______________________________________________
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
