[
https://issues.apache.org/jira/browse/MAPREDUCE-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13400230#comment-13400230
]
Radim Kolar commented on MAPREDUCE-4256:
----------------------------------------
This ticket has 2 main points.
1. pluggable resources framework
2. resource scheduling using pluggable resources
If pluggable resources are implemented pluggable contained monitoring is not
needed, you can not do much other things than kill application using too much
resources. Possible extensions can be done by some kind of pluggable policy
what to do if resource consumption is exceeded.
Instead of hacking
hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/Resource.java
in MAPREDUCE-4327 for cpu cores, its better to add general purpose Resource
management.
> Improve resource scheduling
> ---------------------------
>
> Key: MAPREDUCE-4256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4256
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: resourcemanager
> Reporter: Radim Kolar
>
> Currently resource manager supports only Memory resource during container
> allocation.
> I propose following improvements:
> 1. add support for CPU utilization. Node CPU used information can be obtained
> by ResourceCalculatorPlugin.
> 2. add support for custom resources. In node configuration will be something
> like:
> name=node.resource.GPU, value=1 (node has 1 GPU).
> If job will need to use GPU for computation, it will add "GPU=1" requirement
> to its job config and Resource Manager will allocate container on node with
> GPU available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
