[
https://issues.apache.org/jira/browse/YARN-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331102#comment-15331102
]
Carlo Curino commented on YARN-5215:
------------------------------------
Regarding preemptable vs OPPORTUNISTIC, we had this conversation with
[~kkaranasos] and [~asuresh], where containers types could be
{{NON-PREEMPTABLE}}, {{PREEMPTABLE}}, {{OPPORTUNISTIC}}, describing basically
increasing level of how likely is a task to be interrupted.
For {{NON-PREEMPTABLE}} the system does everything it can not to interrupt (bar
physical machine failures), {{PREEMPTABLE}} are containers with a low, but
non-null chance of being interrupted by the scheduler (e.g., allocations above
a queue capacity, but on dedicated resources), and {{OPPORTUNISTIC}} are tasks
that have a high(er) chance of being interrupted as they run on the left-over
capacity from other containers, or on overcommitted resources or, or other
risky forms of resource harvesting (as in this JIRA).
BTW [~elgoiri] I would suggest to turn this in an umbrella JIRA, and separate
the sub-step you list [in the comment above |
https://issues.apache.org/jira/browse/YARN-5215?focusedCommentId=15330649&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15330649].
Some are likely less controversial and can be settled/committed earlier, while
other parts are too "interesting" to go in easily :-)
> Scheduling containers based on external load in the servers
> -----------------------------------------------------------
>
> Key: YARN-5215
> URL: https://issues.apache.org/jira/browse/YARN-5215
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Inigo Goiri
> Attachments: YARN-5215.000.patch, YARN-5215.001.patch
>
>
> Currently YARN runs containers in the servers assuming that they own all the
> resources. The proposal is to use the utilization information in the node and
> the containers to estimate how much is consumed by external processes and
> schedule based on this estimation.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
