[
https://issues.apache.org/jira/browse/MAPREDUCE-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13280049#comment-13280049
]
Radim Kolar commented on MAPREDUCE-4256:
----------------------------------------
Two kind of resources:
1. cluster shared resources - every node in cluster can use them. For example
WAN bandwidth.
2. node local resources - for example GPU card.
these resources will be configured using standard hadoop configuration
framework.
mapreduce.resources.cluster.available=resourcename:float,... (this option will
be used by resourcemanager).
mapreduce.resources.node.available=resourcename:float (for use by nodemanager).
Node manager will report available resources to resourcemanager
job config:
mapreduce.resources.mapper= list of resources needed by one mapper
mapreduce.resources.reducer= list of resources needed by one reducer
if same resource is configured per node and per cluster then per node version
is preferred.
> Improve resource scheduling
> ---------------------------
>
> Key: MAPREDUCE-4256
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4256
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: resourcemanager
> Reporter: Radim Kolar
> Fix For: 2.0.0, 3.0.0
>
>
> Currently resource manager supports only Memory resource during container
> allocation.
> I propose following improvements:
> 1. add support for CPU utilization. Node CPU used information can be obtained
> by ResourceCalculatorPlugin.
> 2. add support for custom resources. In node configuration will be something
> like:
> name=node.resource.GPU, value=1 (node has 1 GPU).
> If job will need to use GPU for computation, it will add "GPU=1" requirement
> to its job config and Resource Manager will allocate container on node with
> GPU available.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
