[
https://issues.apache.org/jira/browse/YARN-4597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun Suresh updated YARN-4597:
------------------------------
Attachment: YARN-4597.013.patch
Thanks again for the review [~kkaranasos],
I've updated the patch based on your suggestions except the following:
1.
bq. ..the fields of the OpportunisticContainersStatus() can still be updated
during the getOpportunisticContainersStatus(). To avoid synchronization, we
could set the fields using an event, and then in the
getOpportunisticContainersStatus() we would just return the object.
Given that the opportunisticContainerStatus is meant for reporting, I think we
can live with minor temporary inconsistencies. An event is probably an overkill.
2.
bq. In the SchedulerNode, I still suggest to put the ++numContainers and the
--numContainers inside the if statements. If I remember well, these fields are
used for the web UI, so there will be a disconnect between the resources used
(referring only to guaranteed containers) and the number of containers
(referring to both guaranteed and opportunistic at the moment). The stats for
the opportunistic containers are carried by the opportunisticContainersStatus,
so we are good with reporting them too.
I still feel numContainers SHOULD include the opportunistic containers. Else,
we should change the 'numContainers' field name to 'numGuaranteedContainers' -
but I am less inclined to make anymore changes in the RM for this patch. I am
actually of the opinion that since users can see the actual opp. container
count, by virtue of YARN-2995, the total count would be more valuable than just
the guaranteed container count. Also in the future, we might have other
container types, and I would not want a specific counter on the SchedulerNode
for each of these.
> Add SCHEDULE to NM container lifecycle
> --------------------------------------
>
> Key: YARN-4597
> URL: https://issues.apache.org/jira/browse/YARN-4597
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Reporter: Chris Douglas
> Assignee: Arun Suresh
> Labels: oct16-hard
> Attachments: YARN-4597.001.patch, YARN-4597.002.patch,
> YARN-4597.003.patch, YARN-4597.004.patch, YARN-4597.005.patch,
> YARN-4597.006.patch, YARN-4597.007.patch, YARN-4597.008.patch,
> YARN-4597.009.patch, YARN-4597.010.patch, YARN-4597.011.patch,
> YARN-4597.012.patch, YARN-4597.013.patch
>
>
> Currently, the NM immediately launches containers after resource
> localization. Several features could be more cleanly implemented if the NM
> included a separate stage for reserving resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
