[
https://issues.apache.org/jira/browse/OOZIE-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545259#comment-15545259
]
Peter Bacsko edited comment on OOZIE-2686 at 10/4/16 2:36 PM:
--------------------------------------------------------------
The following async actions require MapReduce jars (in parentheses - the class
which caused {{NoClassDefFoundError}}):
- Pig ({{org.apache.hadoop.mapred.JobConf}})
- Hive ({{org.apache.hadoop.mapreduce.TaskAttemptContext}})
- MapReduce ({{org.apache.hadoop.mapred.JobConf}})
- DistCp ({{org.apache.hadoop.mapreduce.Cluster}})
- Spark ({{org.apache.hadoop.mapred.TextInputFormat}})
Async actions that don't need MR jars:
- Java (only if we want to run something which is MR-specific)
- Sqoop
- Shell
was (Author: pbacsko):
The following async actions require MapReduce jars (in parentheses - the class
which caused {{NoClassDefFoundError}}):
- Pig ({{org.apache.hadoop.mapred.JobConf}})
- Hive ({{org.apache.hadoop.mapreduce.TaskAttemptContext}})
- MapReduce ({{org.apache.hadoop.mapred.JobConf}})
- DistCp ({{org.apache.hadoop.mapreduce.Cluster}})
Async actions that don't need MR jars:
- Java (only if we want to run something which is MR-specific)
- Sqoop
- Shell
> Find places where to add MR specific classes/configs instead of handling them
> in JavaActionExecutor
> ---------------------------------------------------------------------------------------------------
>
> Key: OOZIE-2686
> URL: https://issues.apache.org/jira/browse/OOZIE-2686
> Project: Oozie
> Issue Type: Sub-task
> Reporter: Peter Cseh
>
> While work is in progress on OYA, we've introduced some MR specific things in
> JavaActionExecutor, which are probably not needed for every action.
> Find the appropriate place for these:
> * remove from JavaActionExecutor.createBaseHadoopConf:
> {code}
> // FIXME - think about this!
> Element e = actionXml.getChild("config-class", ns);
> if (e != null) {
> conf.set(LauncherMapper.OOZIE_ACTION_CONFIG_CLASS,
> e.getTextTrim());
> }
> {code}
> * MR jars are added in JavaActionExecutor.createAppSubmissionContext
> {code}
> ClasspathUtils.addMapReduceToClasspath(env, launcherJobConf);
> {code}
> * logging setup in JavaActionExecutor.createAppSubmissionContext and in
> LauncherAM has to be worked on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
