[
https://issues.apache.org/jira/browse/YARN-6706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16045229#comment-16045229
]
Haibo Chen commented on YARN-6706:
----------------------------------
This patch does the following:
1) Changes the ContainerScheduler.scheduleContainer() logic
if the container is guaranteed, we first put it into the guaranteed container
queue, then do a pass-through guaranteed queue followed by opportunistic queue
to start as many containers as possible. Finally, if the container remains in
the guaranteed queue, we know we need to kill some opportunistic container
if the container is opportunistic, we first do a pass through pass-through
guaranteed queue followed by opportunistic queue to start as many containers as
possible. Then we try to enqueue the opportunistic container, which may fail if
the maxOppQueue length is reached. Lastly, we may be able to start it if there
happen to be some resources remaining. This way, we slightly change the
semantic of maxOppQueueLength in cases where oversubscription is enabled. We
enqueue an opportunistic container when neither there is unallocated resource
nor can it be started due to oversubscription.
2) When maxOppQUeueLength is set to 0 or a negative value, containers are no
longer unconditionally started.
3) Adds a new unit test in TestContainerSchedulerQueuing to test the case where
the OppQueue length limit is reached with too many opportunistic container
requests.
4) Add a new method resourceAvailableToStartContainer() which we could extend
to include oversubscription logic.
> Refactor ContainerScheduler to make oversubscription change easier
> ------------------------------------------------------------------
>
> Key: YARN-6706
> URL: https://issues.apache.org/jira/browse/YARN-6706
> Project: Hadoop YARN
> Issue Type: Sub-task
> Affects Versions: 3.0.0-alpha3
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Attachments: YARN-6706-YARN-1011.00.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
