[
https://issues.apache.org/jira/browse/OOZIE-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444432#comment-13444432
]
Robert Kanter commented on OOZIE-654:
-------------------------------------
- The code that looking at the scheme://HOST:PORT stuff doesn't actually call
setJar(); that happens later in the MapReduceMain class. All this code does is
resolve the uber jar url. And because its writing to the same property that
its reading, if the uberjar already has scheme://HOST:PORT, then there's
nothing to resolve and we can leave it as is.
- Yes, the createLauncherConf() method is always called after the
setupActionConf() method; this happens in JavaActionExecutor.submitLauncher().
I could do what you said with a private method, but that's going to parse
through the <configuration> and <job-xml> twice to do the same thing; wouldn't
it be better to only do it once?
> Provide a way to use 'uber' jars with Oozie MR actions
> ------------------------------------------------------
>
> Key: OOZIE-654
> URL: https://issues.apache.org/jira/browse/OOZIE-654
> Project: Oozie
> Issue Type: Improvement
> Reporter: Harsh J
> Assignee: Robert Kanter
> Priority: Minor
> Attachments: OOZIE-654.patch, OOZIE-654.patch, OOZIE-654.patch
>
>
> Right now, say if you have a custom MR code in a jar that has a {{lib/}}
> folder inside which carries more dependent jars (a structure known as 'uber'
> jars), and you submit the job via a regular 'hadoop jar' command, these
> lib/*.jars get picked up by the framework because the supplied jar is
> specified explicitly via conf.setJarByClass or conf.setJar. That is, if this
> user uber jar goes to the JT as the mapred.jar, then it is handled by the
> framework properly and the lib/*.jars are all considered and placed on the
> classpath.
> Distributed cache jars do not have this effect, and that is cause the MR
> framework does not consider them as uber jars and does not extract and use
> their internal lib/ directories.
> We should have a way in oozie to let users promote one of their jars as uber
> jars, as an option.
> Proposal: Have an optional oozie-prefixed config, or an optional element in
> the MR action XML, that lets user specify what class should be loaded to be
> set as setJarByClass(...). This will have to be a class available in the
> higher level of the uber jar (not under lib/) but can be any class inside the
> targeted jar really (just not from a jar under lib/). We then set this as
> jobConf.setJarByClass(loadedCls), and then run the job.
> Thoughts?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
