[
https://issues.apache.org/jira/browse/YARN-4105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated YARN-4105:
-----------------------------
Fix Version/s: 2.8.0
> Capacity Scheduler headroom for DRF is wrong
> --------------------------------------------
>
> Key: YARN-4105
> URL: https://issues.apache.org/jira/browse/YARN-4105
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
> Affects Versions: 2.6.0
> Reporter: Chang Li
> Assignee: Chang Li
> Fix For: 2.8.0, 2.7.2, 3.0.0-alpha1
>
> Attachments: YARN-4105.2.patch, YARN-4105.3.patch, YARN-4105.4.patch,
> YARN-4105.patch
>
>
> relate to the problem discussed in YARN-1857. But the min method is flawed
> when we are using DRC. Have run into a real scenario in production where
> queueCapacity: <memory:1056256, vCores:3750>, qconsumed: <memory:1054720,
> vCores:361>, consumed: <memory:125952, vCores:170> limit: <memory:214016,
> vCores:755>. headRoom calculation returns 88064 where there is only 1536
> left in the queue because DRC effectively compare by vcores. It then caused
> deadlock because RMcontainer allocator thought there is still space for
> mapper and won't preempt a reducer in a full queue to schedule a mapper.
> Propose fix with componentwiseMin.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
