[
https://issues.apache.org/jira/browse/YARN-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15311530#comment-15311530
]
Arun Suresh edited comment on YARN-5171 at 6/2/16 1:10 AM:
-----------------------------------------------------------
Thanks for the patch [~elgoiri]..
The general approach looks good, We would also need to ensure the following
though :
# Since the container is created outside the RM, the {{attemptResourceUsage}}
field in the {{SchedulerApplicationAttempt}} is not incremented (This is done
by the specific schedulers and is found in the specific implementation of the
SchedulerApplicationAttempt). I guess we should have a separate
{{attemptResourceUsageOpportunistic}} field introduced in the
SchedulerApplicationAttempt class, that is incremented when the
DistributedSchedulingService creates the RMConatiner.
# Similarly, we should modify the {{completeContainer()}} method in the
{{AbstractYarnScheduler}} to also decrement the
{{attemptResourceUsageOpportunistic}} of the corresponding app attempt if the
completed container was opportunistic (we can get that from the
{{ContainerStatus}}). We don't have to worry about NOT decrementing
{{attemptResourceUsage}}, since that is done in the specific Scheduler, and we
are modifying the {{AbstractYarnScheduler::completedContainer()}}.
was (Author: asuresh):
Thanks for the patch [~elgoiri]..
The general approach looks good, We would also need to ensure the following
though :
# Since the container is created outside the RM, the {{attemptResourceUsage}}
field in the {{SchedulerApplicationAttempt}} is not incremented (This is done
by the specific schedulers and is found in the specific implementation of the
SchedulerApplicationAttempt). I guess we should have a separate
{{attemptResourceUsageOpportunistic}} field introduced in the
SchedulerApplicationAttempt class, that is incremented when the
DistributedSchedulingService creates the RMConatiner.
# Similarly, we should modify the {{completeContainer()}} method in the
{{AbstractYarnScheduler}} to also decrement the
{{attemptResourceUsageOpportunistic}} of the corresponding app attempt if the
completed container was opportunistic (we can get that from the
{{ContainerStatus}})
> Extend DistributedSchedulerProtocol to notify RM of containers allocated by
> the Node
> ------------------------------------------------------------------------------------
>
> Key: YARN-5171
> URL: https://issues.apache.org/jira/browse/YARN-5171
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Arun Suresh
> Assignee: Inigo Goiri
> Attachments: YARN-5171.000.patch, YARN-5171.001.patch
>
>
> Currently, the RM does not know about Containers allocated by the
> OpportunisticContainerAllocator on the NM. This JIRA proposes to extend the
> Distributed Scheduler request interceptor and the protocol to notify the RM
> of new containers as and when they are allocated at the NM. The
> {{RMContainer}} should also be extended to expose the {{ExecutionType}} of
> the container.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
