[
https://issues.apache.org/jira/browse/OOZIE-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444193#comment-13444193
]
Alejandro Abdelnur commented on OOZIE-654:
------------------------------------------
* please trim the license notice trailing spaces
* please add a comment in the pom.xml exclusion what versions of Hadoop
uber-jars in HDFS are supported (1.2.0+ & 2.2.0+)
* the uberjar should also be set in the launcher config, as the client
invocation may need the classes/jars in the uberjar.
* if the uberjar path is relative, it should resolve to the WF app.
* if the uberjar path is absolute (with no authority), it should resolve to the
<name-node> (which must always be present, right). If I read the code
correctly, I think you are doing this already.
> Provide a way to use 'uber' jars with Oozie MR actions
> ------------------------------------------------------
>
> Key: OOZIE-654
> URL: https://issues.apache.org/jira/browse/OOZIE-654
> Project: Oozie
> Issue Type: Improvement
> Reporter: Harsh J
> Assignee: Robert Kanter
> Priority: Minor
> Attachments: OOZIE-654.patch, OOZIE-654.patch
>
>
> Right now, say if you have a custom MR code in a jar that has a {{lib/}}
> folder inside which carries more dependent jars (a structure known as 'uber'
> jars), and you submit the job via a regular 'hadoop jar' command, these
> lib/*.jars get picked up by the framework because the supplied jar is
> specified explicitly via conf.setJarByClass or conf.setJar. That is, if this
> user uber jar goes to the JT as the mapred.jar, then it is handled by the
> framework properly and the lib/*.jars are all considered and placed on the
> classpath.
> Distributed cache jars do not have this effect, and that is cause the MR
> framework does not consider them as uber jars and does not extract and use
> their internal lib/ directories.
> We should have a way in oozie to let users promote one of their jars as uber
> jars, as an option.
> Proposal: Have an optional oozie-prefixed config, or an optional element in
> the MR action XML, that lets user specify what class should be loaded to be
> set as setJarByClass(...). This will have to be a class available in the
> higher level of the uber jar (not under lib/) but can be any class inside the
> targeted jar really (just not from a jar under lib/). We then set this as
> jobConf.setJarByClass(loadedCls), and then run the job.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
