[
https://issues.apache.org/jira/browse/YARN-9515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456905#comment-17456905
]
Akira Ajisaka commented on YARN-9515:
-------------------------------------
According to
[https://github.com/apache/hadoop/blob/d59890404611629d364c39537e6a0a53808403e1/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java#L834-L841]
{code:java}
// - When doing IGNORE_PARTITION_EXCLUSIVITY allocation, we will not
respect
// queue's max capacity, queue's max capacity on the partition will be
// considered to be 100%. Which is a queue can use all resource in the
// partition.
// Doing this because: for non-exclusive allocation, we make sure there's
// idle resource on the partition, to avoid wastage, such resource will be
// leveraged as much as we can, and preemption policy will reclaim it back
// when partitioned-resource-request comes back.
{code}
this issue looks as expected, and I think it won't be fixed.
> Non-exclusive labels do not respect user-limit-factor/max-capacity
> ------------------------------------------------------------------
>
> Key: YARN-9515
> URL: https://issues.apache.org/jira/browse/YARN-9515
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
> Affects Versions: 2.8.5
> Reporter: Brandon Scheller
> Priority: Major
>
> When using a cluster with non-exclusive labels. Queues will only respect
> user-limit-factor for allocations to the default partition. Non-exclusive
> label allocations will not consider these factors allowing the labeled
> partitions capacity to be completely used up.
> To reproduce this, consider this example.
> Cluster contains 1 non-exclusive nodelabel: *APPMASTER*
> "test" queue has access to all labels: *
> "test" queue has capacity/max-capacity: 20
> "test" queue has user-limit-factor: 0.1
> Job is submitted to "test" queue with label: *APPMASTER* only on its
> appMaster container request, and no-label for all other containers.
> user-limit-factor and capacity will not be respected for the job on
> allocations to the *APPMASTER* partition causing the single job to attempt to
> use the entire capacity of the *APPMASTER* partition through non-exclusive
> allocations.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
