[ https://issues.apache.org/jira/browse/YARN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730966#comment-14730966 ]
Hudson commented on YARN-4105: ------------------------------ FAILURE: Integrated in Hadoop-trunk-Commit #8403 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8403/]) YARN-4105. Capacity Scheduler headroom for DRF is wrong. Contributed by Chang Li (jlowe: rev 6eaca2e3634a88dc55689e8960352d6248c424d9) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestCapacityScheduler.java > Capacity Scheduler headroom for DRF is wrong > -------------------------------------------- > > Key: YARN-4105 > URL: https://issues.apache.org/jira/browse/YARN-4105 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler > Affects Versions: 2.6.0 > Reporter: Chang Li > Assignee: Chang Li > Fix For: 2.7.2 > > Attachments: YARN-4105.2.patch, YARN-4105.3.patch, YARN-4105.4.patch, > YARN-4105.patch > > > relate to the problem discussed in YARN-1857. But the min method is flawed > when we are using DRC. Have run into a real scenario in production where > queueCapacity: <memory:1056256, vCores:3750>, qconsumed: <memory:1054720, > vCores:361>, consumed: <memory:125952, vCores:170> limit: <memory:214016, > vCores:755>. headRoom calculation returns 88064 where there is only 1536 > left in the queue because DRC effectively compare by vcores. It then caused > deadlock because RMcontainer allocator thought there is still space for > mapper and won't preempt a reducer in a full queue to schedule a mapper. > Propose fix with componentwiseMin. -- This message was sent by Atlassian JIRA (v6.3.4#6332)