Karthik Kambatla commented on YARN-2618:

Thanks Wei. Comments:
# I think we should avoid adding new methods to BuilderUtils. 
Resource.newInstance does the necessary job. But for the number of places 
BuilderUtils is used, we want to get rid of it. 
# Add vdisks to ResourceType. 
# DominantResourceCalculator#getResourceAsValue: We should probably leave the 
method's name as is, and change the boolean argument to either an int or the 
ResourceType itself. 
# In DRC#computeAvailableContainers, we should add check-for-zero for memory 
and cpu as well. Similarly, DRC#ratio should have zero-checks for memory and 
cpu as well.

bq. Capacity Scheduler already support multi-dimension resource by 
DominateResourceCalculator and it should work when DRC updated to support disk.
I haven't looked at the CS code much, but it appears the scheduling decisions 
use DRC#divide which only looks at memory. [~leftnoteasy] - any thoughts here? 

> Avoid over-allocation of disk resources
> ---------------------------------------
>                 Key: YARN-2618
>                 URL: https://issues.apache.org/jira/browse/YARN-2618
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wei Yan
>            Assignee: Wei Yan
>         Attachments: YARN-2618-1.patch, YARN-2618-2.patch
> Subtask of YARN-2139. 
> This should include
> - Add API support for introducing disk I/O as the 3rd type resource.
> - NM should report this information to the RM
> - RM should consider this to avoid over-allocation

This message was sent by Atlassian JIRA

Reply via email to