[
https://issues.apache.org/jira/browse/YARN-9351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Juanjuan Tian updated YARN-9351:
---------------------------------
Summary: user can't use total resources of one partition even when
yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100
(was: user can't use total resources of one partition even
yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100 )
> user can't use total resources of one partition even when
> yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-9351
> URL: https://issues.apache.org/jira/browse/YARN-9351
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 3.1.2
> Reporter: Juanjuan Tian
> Assignee: Juanjuan Tian
> Priority: Major
>
> if we configure queue capacity in absolute term, users can't use total
> resource of one partition even
> yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent is set to 100
> for example there are two partition A,B, partition A has (120G memory,30
> vcores), and partition B has (180G memory,60 vcores), and Queue Prod is
> configured with (75G memory, 25 vcores) partition A resource, like
> yarn.scheduler.capacity.root.Prod.accessible-node-labels.A.capacity=[memory=75Gi,vcores=25],
> and
> yarn.scheduler.capacity.root.Prod.accessible-node-labels.A.maximum-capacity=[memory=120Gi,vcores=30]
> yarn.scheduler.capacity.root.Prod.minimum-user-limit-percent=100, and at one
> point the used resource of queue Prod is (90G memory,10 vcores), at this time
> even though yarn.scheduler.capacity.<queue-path>.minimum-user-limit-percent
> is set to 100 , users in queue A can't get more resource.
>
> the reason for this is that when {color:#d04437}*computeUserLimit*{color},
> partitionResource is used for comparing consumed, queueCapacity, so in the
> example (75G memory, 25 vcores) is the user limit.
> Resource currentCapacity = Resources.lessThan(resourceCalculator,
> partitionResource, consumed, queueCapacity)
> ? queueCapacity
> : Resources.add(consumed, required);
> Resource userLimitResource = Resources.max(resourceCalculator,
> partitionResource,Resources.divideAndCeil(resourceCalculator, resourceUsed,
> usersSummedByWeight),Resources.divideAndCeil(resourceCalculator,Resources.multiplyAndRoundDown(currentCapacity,
> getUserLimit()),100));
>
> but when *{color:#d04437}canAssignToUser{color}* =
> Resources.greaterThan(resourceCalculator, clusterResource,
> user.getUsed(nodePartition), limit)
> *{color:#d04437}clusterResource{color}* {color:#333333}is used for for
> comparing *used and limit, the result is false.*{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
