[
https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568441#comment-16568441
]
Eric Payne commented on YARN-8509:
----------------------------------
[~Zian Chen], [~sunilg], and others: Can you please clarify what problem we are
trying to address with this JIRA? The description says:
{quote}
This will prevent queue from taking more pending resource to achieve queue
balance after all queue satisfied with its ideal allocation.
{quote}
1) I assume this is referring to the feature for property
{{preemption.preemption-to-balance-queue-after-satisfied.enabled}} which was
added by YARN-8379
2) If so, then queue balancing seems to be working, except that in my testing
with balancing enabled (in trunk build--NOT with this patch), a lot of
unnecessary preemption also happens. Can you please clarify?
> Fix UserLimit calculation for preemption to balance scenario after queue
> satisfied
> ------------------------------------------------------------------------------------
>
> Key: YARN-8509
> URL: https://issues.apache.org/jira/browse/YARN-8509
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Zian Chen
> Assignee: Zian Chen
> Priority: Major
> Attachments: YARN-8509.001.patch, YARN-8509.002.patch,
> YARN-8509.003.patch
>
>
> In LeafQueue#getTotalPendingResourcesConsideringUserLimit, we calculate total
> pending resource based on user-limit percent and user-limit factor which will
> cap pending resource for each user to the minimum of user-limit pending and
> actual pending. This will prevent queue from taking more pending resource to
> achieve queue balance after all queue satisfied with its ideal allocation.
>
> We need to change the logic to let queue pending can go beyond userlimit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
