Karthik Kambatla commented on YARN-3891:

In these cases, MR should preempt reducers and spawn mappers. A bug in headroom 
calculation (YARN-3485) leads to this MR issue. Could this be the same issue? 
MAPREDUCE-6302 is related. 

> Apps are starved for resource due to default FairSharePolicy in Fair Scheduler
> ------------------------------------------------------------------------------
>                 Key: YARN-3891
>                 URL: https://issues.apache.org/jira/browse/YARN-3891
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Siqi Li
>            Priority: Critical
> Based on FairSharePolicy, FSLeafQueue will try to schedule an app that has 
> the lowest resource consumption first.
> However, we have seen some apps that have several thousand reducers stuck at 
> waiting for some newly scheduled mappers to launch. However, it would be very 
> difficult for such apps to get resources if there are many other jobs with 
> low resource consumption and the queue is full.
> If the queue keeps getting new applications, then the stuck reducers would 
> never finish.

This message was sent by Atlassian JIRA

Reply via email to