[
https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512924#comment-16512924
]
Haibo Chen commented on YARN-8250:
----------------------------------
Thanks [~kkaranasos] for your comments and suggestions!
{quote}I understand that this might lead to more opportunistic containers
killed than might be needed, but for sure it will be a good first version (and
much better than what we have today). At the same time, if we are not very
aggressive with starting opportunistic containers, it should not lead to too
many containers killed.
{quote}
Indeed. This is less of an issue if we can be less aggressive to start
opportunistic containers.
{quote}Why not keep existing behavior and add the new one too?
{quote}
Sure. Starting opportunistic containers at container schedule event is very
much similar to starting opportunistic containers at container finish event, so
I'd make the change as you suggested for both container finish and schedule
event. This does allow us to tune how opportunistic containers are launched
without introducing a different implementation.
We'll proceed to experiment with this idea in YARN-8427, and open jiras in the
future if needed for any issue left unaddressed.
Thanks [~asuresh], [~leftnoteasy] [~kkaranasos] for sharing your thoughts,
inputs and reviews!
> Create another implementation of ContainerScheduler to support NM
> overallocation
> --------------------------------------------------------------------------------
>
> Key: YARN-8250
> URL: https://issues.apache.org/jira/browse/YARN-8250
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Major
> Attachments: YARN-8250-YARN-1011.00.patch,
> YARN-8250-YARN-1011.01.patch, YARN-8250-YARN-1011.02.patch
>
>
> YARN-6675 adds NM over-allocation support by modifying the existing
> ContainerScheduler and providing a utilizationBased resource tracker.
> However, the implementation adds a lot of complexity to ContainerScheduler,
> and future tweak of over-allocation strategy based on how much containers
> have been launched is even more complicated.
> As such, this Jira proposes a new ContainerScheduler that always launch
> guaranteed containers immediately and queues opportunistic containers. It
> relies on a periodical check to launch opportunistic containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
