[
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14207172#comment-14207172
]
Wangda Tan commented on YARN-2056:
----------------------------------
Hi [~eepayne]
Thanks for update,
Major comments:
1. Regarding the new requirement you added:
bq. The current patch only allows the disable queue preemption flag to be set
on leaf queues. However, after discussing his internally, we need to be able to
have leaf queues inherit this property from their parent.
I think the feature is make sense, but I don't think we can really achieve it
today. In your example, let's say root has a and b, a has a1/a2 and b has
b1/b2. It maybe no problem to set a is non-preemptable but a1/a2 are
preemptable, the existing ideal_capacity calculation algorithm will consider
this and mark containers will be preempted in a1/a2 as what you expected.
However, you cannot say, in this case, b cannot preempt resource from a.
Because,
When a container preempted, the resource is available for everyone to use. Like
the resource freed after preempt a container from a2 that doesn't mean the
resource is dedicated to a1 to use only.
So the statements are not always true:
{code}
A should not be preemptable
A1 and A2 should be able to preempt each other
{code}
So my opinion is not do it now since we don't have corresponding logic in CS
side for this feature.
Minor comments:
1. getUnderservedQueues the name is a little confusing me, it should be
getMostUnderServedQueues
2. I would suggest to create a method in
CapacitySchedulerConfiguration.getPreemptionEnabled(queue)
3. In {{getUnderservedQueues}}, you can simple use tqComparator.compare(q0, q1)
instead of calculate idealPctGuaranteed
4. it's better to rename idealPctGuaranteed -> caculate(or
get)IdealPctGuaranteed.
For tests, I'll review it after we can get decision about my concern in *major
comments*.
Wangda
> Disable preemption at Queue level
> ---------------------------------
>
> Key: YARN-2056
> URL: https://issues.apache.org/jira/browse/YARN-2056
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Mayank Bansal
> Assignee: Eric Payne
> Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt,
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt,
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt,
> YARN-2056.201409232329.txt, YARN-2056.201409242210.txt,
> YARN-2056.201410132225.txt, YARN-2056.201410141330.txt,
> YARN-2056.201410232244.txt, YARN-2056.201410311746.txt,
> YARN-2056.201411041635.txt, YARN-2056.201411072153.txt
>
>
> We need to be able to disable preemption at individual queue level
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
