[ https://issues.apache.org/jira/browse/YARN-10821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17364966#comment-17364966 ]
Eric Payne commented on YARN-10821: ----------------------------------- [~gandras], I am running behind. I will try to get back to this early next week. > User limit is not calculated as per definition for preemption > ------------------------------------------------------------- > > Key: YARN-10821 > URL: https://issues.apache.org/jira/browse/YARN-10821 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler > Reporter: Andras Gyori > Assignee: Andras Gyori > Priority: Major > Attachments: YARN-10821.001.patch > > > Minimum user limit percent (MULP) is a soft limit by definition. Preemption > uses pending resources to determine the resources needed by a queue, which is > calculated in LeafQueue#getTotalPendingResourcesConsideringUserLimit. This > method involves headroom calculated by UsersManager#computeUserLimit. > However, the pending resources for preemption are limited in an unexpected > fashion. > * In LeafQueue#getUserAMResourceLimitPerPartition an effective userLimit is > calculated first: > {code:java} > float effectiveUserLimit = Math.max(usersManager.getUserLimit() / 100.0f, > 1.0f / Math.max(getAbstractUsersManager().getNumActiveUsers(), 1)); > {code} > * In UsersManager#computeUserLimit the userLimit is calculated as is > (currentCapacity * userLimit) > {code:java} > Resource userLimitResource = Resources.max(resourceCalculator, > partitionResource, > Resources.divideAndCeil(resourceCalculator, resourceUsed, > usersSummedByWeight), > Resources.divideAndCeil(resourceCalculator, > Resources.multiplyAndRoundDown(currentCapacity, getUserLimit()), > 100)); > {code} > The fewer users occupying the queue, the more prevalent and outstanding this > effect will be in preemption. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org