[
https://issues.apache.org/jira/browse/YARN-8509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16568258#comment-16568258
]
Eric Payne commented on YARN-8509:
----------------------------------
[~Zian Chen]. Unfortunately, this code cannot go in as it is. This change
causes very many unnecessary preemptions, which causes a lot of lost work.
{code}
Resource minpendingConsideringUserLimit = Resources.componentwiseMin(
userNameToHeadroom.get(userName), pending);
Resources.addTo(totalPendingConsideringUserLimit,
- minpendingConsideringUserLimit);
+ pending);
Resources.subtractFrom(userNameToHeadroom.get(userName),
minpendingConsideringUserLimit);
}
{code}
The purpose of doing a min between the user's pending and the user's headroom
is to prevent a preemption that cannot be given to the user because the user
has reached its headroom limit.
> Fix UserLimit calculation for preemption to balance scenario after queue
> satisfied
> ------------------------------------------------------------------------------------
>
> Key: YARN-8509
> URL: https://issues.apache.org/jira/browse/YARN-8509
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
> Reporter: Zian Chen
> Assignee: Zian Chen
> Priority: Major
> Attachments: YARN-8509.001.patch, YARN-8509.002.patch,
> YARN-8509.003.patch
>
>
> In LeafQueue#getTotalPendingResourcesConsideringUserLimit, we calculate total
> pending resource based on user-limit percent and user-limit factor which will
> cap pending resource for each user to the minimum of user-limit pending and
> actual pending. This will prevent queue from taking more pending resource to
> achieve queue balance after all queue satisfied with its ideal allocation.
>
> We need to change the logic to let queue pending can go beyond userlimit.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
