On 3/5/08 7:12 PM, "Camille Coti" <c...@lri.fr> wrote:
> Ralph Castain wrote:
>> On 3/5/08 3:31 PM, "Aurélien Bouteiller" <boute...@eecs.utk.edu> wrote:
>>
>>
>>> From : Camille Coti <c...@eecs.utk.edu>
>>> Date : 5 mars 2008 17:26:32 HNE (ÃUA)
>>> to : de...@open-mpi.org
>>> Objet : orte_job_data
>>>
>>>
>>> Hello,
>>>
>>> I had a look at how the job data are kept during the life cycle of a
>>> job.
>>>
>>> The orte_job_data pointer array contains two elements:
>>> addr[0] is filled during rte_init().
>>> addr[1] is filled by PLM at setup time and contains the map on
>>> which the job is spawned.
>>>
>>> What does the first entry correspond to?
>>>
>>
>> The first element in the array is for the daemon job. All subsequent entries
>> correspond to applications.
>>
>>
>>
>>> Besides, when I dump the map contained in the second element during
>>> the execution of my application (ie, after PLM launched the job), each
>>> node entry contains: "Daemon launched: False". Is this expected?
>>>
>>
>> Yes, that is expected - we don't bother to update that field in the job map
>> when we launch the daemons for an application. However, the next time the
>> job map is retrieved - say to map a comm_spawn'd application - then we do
>> fill it in. The get_job_map function will check the daemon's job data object
>> to see if a daemon has been launched on each node and fill-in the field.
>>
>> Reason we don't automatically update that field as the daemons launch is
>> that we are looking ahead to when we minimize memory usage. When that
>> happens, we probably won't retain the map at all - we'll reconstruct
>> whatever is needed from the minimal stored set of data when it is needed,
>> and then dump it when done.
>>
>> So, since we'll probably have to regenerate the map anyway, we don't bother
>> to maintain the daemon spawned field. We just fill it in when we retrieve a
>> map. If you want to see the filled-in values, call orte_rmaps.get_job_map
>> and then dump the returned map object.
>>
> Ralph,
>
> Thanks a lot for your answer.
> orte_rmaps.get_job_map gave me what I wanted.
>
> I also need to access the mapping information from any node. However,
> orte_rmaps.get_job_map calls orte_get_job_data_object, which returns
> NULL if the node is not an HNP. In that case, is there an alternative
> way to access the mapping information?
Not at the moment - we knew of no immediate need for that data on the orteds
or the application procs. However, we do plan to add that info in the near
future (probably about two weeks out).
The orteds have all the data since they actually receive a complete process
map in the launch message - they just don't store it or do anything with it
(other than setting up the routed system so they know where to relay
messages). So it is rather trivial for us to just have them hang onto it
(though that does increase their memory footprint, so we will want to do
that carefully).
For application procs, Galen had proposed a couple of months ago that we
create a struct that would hold all the RTE-level data that is currently in
ompi_proc_t (e.g., nodeid - which is already in the RTE layer), and then
just let the ompi_proc_t point at it. This may make it somewhat easier to
support multiple environments where that data gets exchanged in different
ways, and would help to reduce the modex message size. Since the orteds
already have the data, it would be simple enough to just "drop" it into the
proc. This wouldn't increase our current memory footprint since we already
store all that data in the proc - we would just reorganize it a little, more
properly do the data update in the RTE layer, etc.
Won't get to that for a couple of weeks, though - I will be on vacation next
week. In the interim, I'm afraid there really isn't any easy way to get it
outside of the HNP.
>
> Camille
>>
>>
>>> Thanks,
>>> Camille
>>> _______________________________________________
>>> devel mailing list
>>> de...@open-mpi.org
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>
>>
>>
>> _______________________________________________
>> devel mailing list
>> de...@open-mpi.org
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
