[
https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330588#comment-15330588
]
Karthik Kambatla commented on YARN-5215:
----------------------------------------
I am generally supportive of this.
Few questions to clarify the usecase and approach:
# How dynamic does this need to be?
# And, what range of utilization improvements are we targeting here? 60 - 80,
75 - 80?
# What are the characteristics of other workload running on these nodes?
The reason I ask is to see if other approaches would suffice. For instance,
would it be enough to gracefully increase/decrease the resources for Yarn on
each node? i.e., {{yarn.nodemanager.resource.*}}. By graceful, I mean the
decrease succeeds only after the tasks using those resources finish.
> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>
> Key: YARN-5215
> URL: https://issues.apache.org/jira/browse/YARN-5215
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Inigo Goiri
> Attachments: YARN-5215.000.patch, YARN-5215.001.patch
>
>
> Currently YARN runs containers in the servers assuming that they own all the
> resources. The proposal is to use the utilization information in the node and
> the containers to estimate how much is consumed by external processes and
> schedule based on this estimation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
