[
https://issues.apache.org/jira/browse/YARN-3388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485952#comment-14485952
]
Wangda Tan commented on YARN-3388:
----------------------------------
[~nroberts],
Thanks for updating, I took a look at your patch, approach LGTM, but I think
node label should be considered in the same JIRA, when doing allocation under a
labeled node, user-limit checking in the patch is incorrect. Actually
user-limit for exclusive node label is already supported in latest trunk, and
user-limit for non-exclusive node label is contains in YARN-3361, I think after
YARN-3361, user-limit for node label will be in a good shape.
Would you mind to take a look at computeUserLimit method of the patch attached
in YARN-3361?
To support computing consumed-per-partition, User.updateUsageRatio need receive
partition as parameter.
Some other comments:
- consumedRatio -> totalDominateConsumed or some other name, it's better to
make sum of "dominate" consumed in the name
- consumed -> totalDominatedConsumedByPartition.
- It's better to add a test case to make sure allocation locked descripted in
https://issues.apache.org/jira/browse/YARN-3388?focusedCommentId=14376060&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14376060
will not happened.
> Allocation in LeafQueue could get stuck because DRF calculator isn't well
> supported when computing user-limit
> -------------------------------------------------------------------------------------------------------------
>
> Key: YARN-3388
> URL: https://issues.apache.org/jira/browse/YARN-3388
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 2.6.0
> Reporter: Nathan Roberts
> Assignee: Nathan Roberts
> Attachments: YARN-3388-v0.patch, YARN-3388-v1.patch
>
>
> When there are multiple active users in a queue, it should be possible for
> those users to make use of capacity up-to max_capacity (or close). The
> resources should be fairly distributed among the active users in the queue.
> This works pretty well when there is a single resource being scheduled.
> However, when there are multiple resources the situation gets more complex
> and the current algorithm tends to get stuck at Capacity.
> Example illustrated in subsequent comment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
