Karthik Kambatla commented on YARN-3332:

Thanks for filing this and working on the design, Vinod. I like the idea of a 
clean interface to get node and container resource usage info. 

Is there any reason why you think a service architecture is better than it 
being a common library? How much information is shared among the consumers of 
this interface? For instance, both HDFS and YARN would be interested in the 
availability and usage of CPU, memory, disk and network for the entire node. 
Isn't all other information of exclusive interest either? 

Have other questions/comments on the design, but will hold off until we decide 
on service vs library. 

> [Umbrella] Unified Resource Statistics Collection per node
> ----------------------------------------------------------
>                 Key: YARN-3332
>                 URL: https://issues.apache.org/jira/browse/YARN-3332
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: Design - UnifiedResourceStatisticsCollection.pdf
> Today in YARN, NodeManager collects statistics like per container resource 
> usage and overall physical resources available on the machine. Currently this 
> is used internally in YARN by the NodeManager for only a limited usage: 
> automatically determining the capacity of resources on node and enforcing 
> memory usage to what is reserved per container.
> This proposal is to extend the existing architecture and collect statistics 
> for usage b​eyond​ the existing use­cases.
> Proposal attached in comments.

This message was sent by Atlassian JIRA

Reply via email to