[
https://issues.apache.org/jira/browse/MAPREDUCE-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13454989#comment-13454989
]
Alejandro Abdelnur commented on MAPREDUCE-4421:
-----------------------------------------------
Arun - make sense, AMs configs should not pollute YARN config.
Now building on this ...
We are aiming at a YARN deployment which does not know anything about AMs that
run on it. This means the cluster setup should not have AMs site.xml files
deployed in it. This means that the cluster setup should not have AMs JARs
deployed in it.
For one off custom AMs, JARs and config would be all provided by the client on
submission.
For AMs like MapReduce, DistributedShell and widely used AMs in a given
cluster, their JARs and config site.xml files would be in HDFS. Then the 'yarn'
client script should support a '-ampath=' (HDFS path to AM resources) option
and/or a '-amname=' (logical name of the AM resources, a new config file
am-site.xml in the cluster would have this mapping for blessed AMs, as
suggested in my prev comment).
Thoughts?
> Remove dependency on deployed MR jars
> -------------------------------------
>
> Key: MAPREDUCE-4421
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4421
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.0.0-alpha
> Reporter: Arun C Murthy
> Assignee: Vinod Kumar Vavilapalli
>
> Currently MR AM depends on MR jars being deployed on all nodes via implicit
> dependency on YARN_APPLICATION_CLASSPATH.
> We should stop adding mapreduce jars to YARN_APPLICATION_CLASSPATH and,
> probably, just rely on adding a shaded MR jar along with job.jar to the
> dist-cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
