[
https://issues.apache.org/jira/browse/MAPREDUCE-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542840#comment-14542840
]
Jason Lowe commented on MAPREDUCE-5951:
---------------------------------------
bq. MRApps - we are using "*" instead of "job.jar" which should work, but I
wonder if that will be an incompatible behavior change. Jason Lowe, Vinod Kumar
Vavilapalli - what do you think?
I get why the '*' is desirable since the name of the jar being shared and
replacing our job jar might not match the job jar name we expect, although I
think a comment stating why would help explain why we're not using the more
specific job jar name that normally would be expected. That would also help
prevent someone else coming along much later and "fixing" that.
I think we're probably OK with using * specifically for the job jar because we
explicitly use mapreduce.job.jar.unpack.pattern when we unarchive it. That
means we'll only unarchive the classes and lib portions of the archive and not
any other files from it, by default. So by default we shouldn't be picking up
any other jars within the job jar. In theory someone might have packed other
jars into the job jar, modified the unpack pattern property to pick up those
jars, then explicitly set their classpath to pick up only a portion of those
jars or all of them in a specific order. '*' does not guarantee order in any
way, so that could break that scenario. I'm not sure that scenario is likely,
however.
> Add support for the YARN Shared Cache
> -------------------------------------
>
> Key: MAPREDUCE-5951
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5951
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Chris Trezzo
> Assignee: Chris Trezzo
> Labels: BB2015-05-TBR
> Attachments: MAPREDUCE-5951-trunk-v1.patch,
> MAPREDUCE-5951-trunk-v10.patch, MAPREDUCE-5951-trunk-v11.patch,
> MAPREDUCE-5951-trunk-v2.patch, MAPREDUCE-5951-trunk-v3.patch,
> MAPREDUCE-5951-trunk-v4.patch, MAPREDUCE-5951-trunk-v5.patch,
> MAPREDUCE-5951-trunk-v6.patch, MAPREDUCE-5951-trunk-v7.patch,
> MAPREDUCE-5951-trunk-v8.patch, MAPREDUCE-5951-trunk-v9.patch
>
>
> Implement the necessary changes so that the MapReduce application can
> leverage the new YARN shared cache (i.e. YARN-1492).
> Specifically, allow per-job configuration so that MapReduce jobs can specify
> which set of resources they would like to cache (i.e. jobjar, libjars,
> archives, files).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
