[
https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16475011#comment-16475011
]
Haibo Chen commented on YARN-8250:
----------------------------------
My understanding of SHED_QUEUED_CONTAINERS is to notify container scheduler to
get rid of some opportunistic containers being queued. The intent of the new
SCHEDULER_CONTAINERS is to let container scheduler try to launch opportunistic
containers that are currently being queued. A follow-up would be also reusing
SCHEDULER_CONTAINERS to preempt running opportunistic containers. I am not
sure how to best align the two.
The main reasons why we'd like to introduce a new container scheduler are
1) Minimize impact on GUARANTEED containers from over-allocating node with
OPPORTUNISTIC containers. Queuing time of GUARANTEED containers would increase
with more running OPPORTUNISTIC containers, which is the case with
over-allocating. The code as in YARN-6675 gets complicated, Alternatively, we
could launch GUARANTEED containers immediately and reply on cgroup-mechanism
for preemption.
2) avoid aggressive OPPORTUNISTIC container launching. One thing to note is
that in case of over-allocation, we'd rely on the resource utilization metrics
to decide how much resources we can to launch OPPORTUNISTIC containers. The
resource utilization metrics in NM is unfortunately only updated every few
seconds. This can be problematic in that NM could end up with launching too
many OPPORTUNISTIC containers before the metric is updated. The current default
container scheduler launches containers aggressively, which could cause
containers to be launched and killed shortly after. The new container
scheduler only schedule OPPORTUNISTIC containers once whenever the utilization
metric is updated.
It is to my understanding that removing GUARANTEED container queuing would
de-stablize cases like yours where nodes are running with a high utilization,
and scheduling OPPORTUNISTIC containers only every few seconds would delay
launch time in distributed scheduling.
Hence, we created a plug-gable container scheduler so that we can choose to do
things differently without causing issues to existing use cases. The new
container scheduler should probably be named or documented so that it is only
used when over-allocation is enabled.
> Create another implementation of ContainerScheduler to support NM
> overallocation
> --------------------------------------------------------------------------------
>
> Key: YARN-8250
> URL: https://issues.apache.org/jira/browse/YARN-8250
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Haibo Chen
> Assignee: Haibo Chen
> Priority: Major
> Attachments: YARN-8250-YARN-1011.00.patch,
> YARN-8250-YARN-1011.01.patch, YARN-8250-YARN-1011.02.patch
>
>
> YARN-6675 adds NM over-allocation support by modifying the existing
> ContainerScheduler and providing a utilizationBased resource tracker.
> However, the implementation adds a lot of complexity to ContainerScheduler,
> and future tweak of over-allocation strategy based on how much containers
> have been launched is even more complicated.
> As such, this Jira proposes a new ContainerScheduler that always launch
> guaranteed containers immediately and queues opportunistic containers. It
> relies on a periodical check to launch opportunistic containers.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
