Karthik Kambatla commented on YARN-3482:

bq. With this and the containers utilization, we can estimate the utilization 
of external processes.

True, but I fear that will be too conservative. If we go that route, HBase 
RegionServers could grow aggressively and adversely affect resources under 
Yarn. By having an interface for available resources, we ensure Yarn 
aggressively schedules work to claim all available resources. Changing these 
available resources could be through a secure interface admins or a white-list 
of processes can access. 

> Report NM available resources in heartbeat
> ------------------------------------------
>                 Key: YARN-3482
>                 URL: https://issues.apache.org/jira/browse/YARN-3482
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager, resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Inigo Goiri
>   Original Estimate: 504h
>  Remaining Estimate: 504h
> NMs are usually collocated with other processes like HDFS, Impala or HBase. 
> To manage this scenario correctly, YARN should be aware of the actual 
> available resources. The proposal is to have an interface to dynamically 
> change the available resources and report this to the RM in every heartbeat.

This message was sent by Atlassian JIRA

Reply via email to