[
https://issues.apache.org/jira/browse/YARN-8808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16631358#comment-16631358
]
Arun Suresh commented on YARN-8808:
-----------------------------------
bq. The scheduler would see this 'Node' has 10GBs to allocate because that's
what NM tells RM. I believe in this case, YARN should try to fully utilize just
10GBs instead of the whole node (100 GBs), because YARN is entitled to use only
10GBs. If 10GBs is indeed fully utilized, the aggregate container utilization
is 100%, but the nodeUtilization is 10% (Again, node utilization by default is
detected by some plugin on NM side that reads from /proc and sees the remaining
system-wide 90GBs as available). Don't think we shall check if nodeUtilization
is low.
Makes sense.. to be honest, in my testing I had also changed it from
nodeUtilization to aggregateContainerUtilization :) I was just wondering if
there is still a case where we might need to factor in nodeUtilization.
> Use aggregate container utilization instead of node utilization to determine
> resources available for oversubscription
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-8808
> URL: https://issues.apache.org/jira/browse/YARN-8808
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: YARN-1011
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Major
> Attachments: YARN-8088-YARN-1011.01.patch,
> YARN-8808-YARN-1011.00.patch
>
>
> Resource oversubscription should be bound to the amount of the resources that
> can be allocated to containers, hence the allocation threshold should be with
> respect to aggregate container utilization.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
