[
https://issues.apache.org/jira/browse/YARN-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355923#comment-14355923
]
Lei Guo commented on YARN-3332:
-------------------------------
To support customized resources, a quick list about areas we need consider
- resource definition, how NM/RM to understand the resource, this should be
considered as Metrics based
- plug-in framework in NM/agent,
* interface for passing resource information between the plug-in and agent,
this could be another RPC interface, so the plug-in can be based on any language
* interface for loading/trigger plug-in (optional), the reason this
interface as optional because the plug-in could be easy as cron job
- Sample resource collection plug-in for specific resource (or resource set),
this could be some script or Java class depending on the plug-in framework
design
- communication protocol between RM/NM to support customized resource
This topic is related to our proposal in June Hadoop Summit on multiple
dimension scheduling.
> [Umbrella] Unified Resource Statistics Collection per node
> ----------------------------------------------------------
>
> Key: YARN-3332
> URL: https://issues.apache.org/jira/browse/YARN-3332
> Project: Hadoop YARN
> Issue Type: Improvement
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Vinod Kumar Vavilapalli
> Attachments: Design - UnifiedResourceStatisticsCollection.pdf
>
>
> Today in YARN, NodeManager collects statistics like per container resource
> usage and overall physical resources available on the machine. Currently this
> is used internally in YARN by the NodeManager for only a limited usage:
> automatically determining the capacity of resources on node and enforcing
> memory usage to what is reserved per container.
> This proposal is to extend the existing architecture and collect statistics
> for usage beyond the existing usecases.
> Proposal attached in comments.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
