[
https://issues.apache.org/jira/browse/YARN-4462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097879#comment-15097879
]
Tao Jie commented on YARN-4462:
-------------------------------
[~kasha] , Thank you for your comments. I will make some improvements as you
suggested.
I have more consideration about:
{quote}
Also, it would be nice to have a global option, so folks could pick a default
enable/disable based on the number of queues they have.
{quote}
I am not sure if it is necessary to have a global option for enable/disable
preemption from one queue, since I don't think it is a reasonable scenario to
set this default value to non-preemptable.
Now option for allowPreemptionFrom is set to true as default. In most scenario,
we have most queues preemptable and few queues non-preemptable, we just add
option allowPreemptionFrom with value of false for those non-preemptable.
Suppose there is another scenario in which we set global option for
allowPreemptionFrom to true. We should have most queues non-preemptable and
only few(maybe none) queues preemptable. There are very large probability of
failure to find candidate containers to be preempted. And it breaks fairness
among queues.
Also it would become confusing when we config queue hierachy. Now we set
non-preemptable option either in parent queue or in its leaf queue, the leaf
queue is non-preemptable. We try to find preemptable queues from root to leaf
recursively and return when we get non-preemptable. It makes sence. However,
when we set global default value to non-preemptable and set one parent queue or
leaf queue value to preemptable, I would be confusing about whether the leaf
queue should be preemptable or not.
> FairScheduler: Disallow preemption from a queue
> -----------------------------------------------
>
> Key: YARN-4462
> URL: https://issues.apache.org/jira/browse/YARN-4462
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Tao Jie
> Assignee: Tao Jie
> Attachments: YARN-4462.001.patch, YARN-4462.002.patch,
> YARN-4462.003.patch
>
>
> When scheduler preemption is enabled, applications could be preempted if they
> obtain resource over they should take.
> When a mapreduce application is preempted some resource, it just runs slower.
> However, when the preempted application is a long-run service, such as tomcat
> running in slider, the service would fail.
> So we should have a flag for application to indicate the scheduler that those
> application should not be preempted.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
