[
https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485884#comment-14485884
]
Jian Fang commented on MAPREDUCE-6304:
--------------------------------------
I understand your point from on-premise cluster perspective. However, it is not
very practical to manage mapred-site.xml or queue files for users if hadoop is
a service in cloud. As a hadoop developer, you should consider both on-premise
hadoop cluster and hadoop in cloud.
There are many many users for a hadoop cloud service. Usually they launch their
own hadoop clusters in cloud and control their own queue files or
mapred-site.xml. Some of them even run their hadoop jobs on their own gateways
that the hadoop platform provider does not have access to. But the hadoop
service provider may still want to have a mechanism to set up some global
labels for all users to improve their user experiences. For example, a failure
of an application master on a spot instance due to the termination of a spot
instance will cause more trouble than a failure of one MR task. These types of
settings most likely can only be done by hadoop cloud service providers based
on their deep knowledge in their own cloud services.
Or could hadoop provide a mechanism for hadoop providers to extend so that you
only need to specify the labels in YarnRunner in Vanilla hadoop?
> Specifying node labels when submitting MR jobs
> ----------------------------------------------
>
> Key: MAPREDUCE-6304
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6304
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Jian Fang
> Assignee: Naganarasimha G R
>
> Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify
> node labels when submitting MR jobs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
