Varun Vasudev commented on YARN-2448:

The use case that springs to mind is adding support for cpu to map-reduce. 
Currently the map-reduce AM only looks at memory when it is deciding things 
like pre-empting reducers. If we wish to add support for cpu as a resource to 
map-reduce, it needs to consider vcores as well. However, if the YARN scheduler 
if using the DefaultResourceCalculator, which ignores cpu, and the map-reduce 
AM doesn't know this, it leads to inefficient asks and allocations. The aim is 
just to let the AM know which calculator is being used and let the AM go from 
there. Does that make sense?

> RM should expose the name of the ResourceCalculator being used when AMs 
> register
> --------------------------------------------------------------------------------
>                 Key: YARN-2448
>                 URL: https://issues.apache.org/jira/browse/YARN-2448
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: apache-yarn-2448.0.patch, apache-yarn-2448.1.patch
> The RM should expose the name of the ResourceCalculator being used when AMs 
> register, as part of the RegisterApplicationMasterResponse.
> This will allow applications to make better decisions when scheduling. 
> MapReduce for example, only looks at memory when deciding it's scheduling, 
> even though the RM could potentially be using the DominantResourceCalculator.

This message was sent by Atlassian JIRA

Reply via email to