[
https://issues.apache.org/jira/browse/YARN-6670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16046882#comment-16046882
]
Haibo Chen edited comment on YARN-6670 at 6/12/17 6:12 PM:
-----------------------------------------------------------
IIUC, this effectively turns the overallocation threshold from being based on a
utilization snapshot to being based on utilization during a time period. This,
however, can be difficult to implement IMO. To get an accurate picture of CPU
and memory utilization over time, NMs need to check the usage very frequently,
which can be very expensive. In our first phase, we rely on prompt NM
preemption of OPPORTUNISTIC containers and tuning of overallocation threshold
to handle such cases. We could certainly revisit this when we see often spiky
usage
was (Author: haibochen):
IIUC, this effectively turns the overallocation threshold from being based on a
utilization snapshot to being based on utilization during a time period. This,
however, can be difficult to implement IMO. To get an accurate picture of CPU
and memory utilization over time, NMs need to check the usage very frequently,
which can be very expensive. In our first phase, we rely on prompt NM
preemption of OPPORTUNISTIC containers and tuning of overallocation threshold
to handle such cases.
> Add separate NM overallocation thresholds for cpu and memory
> ------------------------------------------------------------
>
> Key: YARN-6670
> URL: https://issues.apache.org/jira/browse/YARN-6670
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Affects Versions: 3.0.0-alpha3
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Attachments: YARN-6670-YARN-1011.00.patch,
> YARN-6670-YARN-1011.01.patch, YARN-6670-YARN-1011.02.patch,
> YARN-6670-YARN-1011.03.patch, YARN-6670-YARN-1011.04.patch,
> YARN-6670-YARN-1011.05.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
