Varun Vasudev commented on YARN-2448:

Sorry the extremely late response. The idea is to allow apps that carry out 
their own scheduling the information they need. MR for example allocates all 
maps and only some reducers. As the maps finish, it allocates more reducers. 
Sometimes due to hanging or stuck maps, it needs to give up some reducers(to 
free up resources to schedule maps). It used to consider only memory when 
deciding how many reducers to give up because it didn't know what the RM was 
considering while scheduling - leading to MAPREDUCE-5279. With this change it 
can give up the right amount of resources.

> RM should expose the resource types considered during scheduling when AMs 
> register
> ----------------------------------------------------------------------------------
>                 Key: YARN-2448
>                 URL: https://issues.apache.org/jira/browse/YARN-2448
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>             Fix For: 2.6.0
>         Attachments: apache-yarn-2448.0.patch, apache-yarn-2448.1.patch, 
> apache-yarn-2448.2.patch
> The RM should expose the name of the ResourceCalculator being used when AMs 
> register, as part of the RegisterApplicationMasterResponse.
> This will allow applications to make better decisions when scheduling. 
> MapReduce for example, only looks at memory when deciding it's scheduling, 
> even though the RM could potentially be using the DominantResourceCalculator.

This message was sent by Atlassian JIRA

Reply via email to