[
https://issues.apache.org/jira/browse/YARN-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087835#comment-16087835
]
Arun Suresh commented on YARN-6808:
-----------------------------------
[~leftnoteasy],
bq. Currently, opportunistic container allocation is too simple to follow what
existing schedulers can do. For example, delayed scheduling, user limit. You
might want to say that there's no guarantee for opportunistic container so we
don't have to follow all of them.
Apologize if I did not articulate it correctly in my earlier comment.. also
kindly do take a look at the v002 patch again.. The Container allocation is
still done by the main configured scheduler, Capacity / Fair. It is just
downgraded to an OC before being sent to the AM. Everything you mentioned:
delayed scheduling, user limit enforcement etc.. all still happen.
bq. If like you said, use this approach to replace existing preemption, I think
this patch is far away from the goal.
Agreed that this is probably not complete. For eg. Like I mentioned, we
probably need to promote OCs to guaranteed containers as and when containers
complete..
I am by no means claiming feature parity. My intention is to run tests and
measure user-perceptible metrics - such as Job runtimes / good put and cluster
utilization. That should give us an objective measure of the viability of the
feature - dont you agree ?
> Allow Schedulers to return OPPORTUNISTIC containers when queues go over
> configured capacity
> -------------------------------------------------------------------------------------------
>
> Key: YARN-6808
> URL: https://issues.apache.org/jira/browse/YARN-6808
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Arun Suresh
> Assignee: Arun Suresh
> Attachments: YARN-6808.001.patch, YARN-6808.002.patch
>
>
> This is based on discussions with [~kasha] and [~kkaranasos].
> Currently, when a Queues goes over capacity, apps on starved queues must wait
> either for containers to complete or for them to be pre-empted by the
> scheduler to get resources.
> This JIRA proposes to allow Schedulers to:
> # Allocate all containers over the configured queue capacity/weight as
> OPPORTUNISTIC.
> # Auto-promote running OPPORTUNISTIC containers of apps as and when their
> GUARANTEED containers complete.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
