[
https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330759#comment-15330759
]
Inigo Goiri commented on YARN-5215:
-----------------------------------
My initial proposal was to add a generic support for external resources.
However, we could also follow the node-level agent approach which could even
show as a unmanaged fake container. That solution is also OK with me.
Going a little bit deeper into the example, in our most extreme scenario, we
would set the guaranteed to 0GB and the opportuinistic to 16GB.
In any case, if we go into preemption, then we should leverage what we are
doing in YARN-1011.
Regarding YARN-5202, they use the concept of preemptable which is pretty much
the same as the OPPORTUNISTIC one. Actually, in our internal deployment right
now, we just assume that everything running on YARN is preemptable and we
preempt the youngest container.
> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>
> Key: YARN-5215
> URL: https://issues.apache.org/jira/browse/YARN-5215
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Inigo Goiri
> Attachments: YARN-5215.000.patch, YARN-5215.001.patch
>
>
> Currently YARN runs containers in the servers assuming that they own all the
> resources. The proposal is to use the utilization information in the node and
> the containers to estimate how much is consumed by external processes and
> schedule based on this estimation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
