[
https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16506607#comment-16506607
]
Arun Suresh commented on YARN-8250:
-----------------------------------
Apologize for the late reply.
{quote}
3) Upon any container completed or finished event, do not try to launch any
container.
4) Introduce a periodic check (in ContainersMonitor thread) that launches
OPPORTUNISTIC container. Ideally, the period is configurable so that the
latency to launch OPPORTUNISTIC containers can be reduced.
{quote}
My only issue is, what would 3) accomplish ? Again, point of opportunistic
containers is to have it start up as fast as possible. If there is capacity at
the time a container completes AND there are no G containers waiting to start,
why not start the first O container in queue ?
Forgive me if my understanding is a bit off, but I am guessing the point of the
JIRA is to ensure G container startup time is not impacted right ? Wouldnt a
simple approach be: Check if container is opportunistic, and if container is to
be killed and if over-allocation is turned on, assume
{{sleep-delay-before-sigkill.ms}} == 0. This will ensure 'kill -9' is called
immediately.
Please do not assume I am trying to 'block' via arbitrary argument :) I am just
saying, we probably need a more data-driven approach while making changes.
[~haibochen], can you provide some numbers to demonstrate the container
start-time deterioration ? Just wanted to know how much of an issue it is. In
our clusters, we pause containers and we've never had much of a problem with G
container startup times, because of the queuing.
[~kkaranasos], thoughts ?
> Create another implementation of ContainerScheduler to support NM
> overallocation
> --------------------------------------------------------------------------------
>
> Key: YARN-8250
> URL: https://issues.apache.org/jira/browse/YARN-8250
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Major
> Attachments: YARN-8250-YARN-1011.00.patch,
> YARN-8250-YARN-1011.01.patch, YARN-8250-YARN-1011.02.patch
>
>
> YARN-6675 adds NM over-allocation support by modifying the existing
> ContainerScheduler and providing a utilizationBased resource tracker.
> However, the implementation adds a lot of complexity to ContainerScheduler,
> and future tweak of over-allocation strategy based on how much containers
> have been launched is even more complicated.
> As such, this Jira proposes a new ContainerScheduler that always launch
> guaranteed containers immediately and queues opportunistic containers. It
> relies on a periodical check to launch opportunistic containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
