[
https://issues.apache.org/jira/browse/OOZIE-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154938#comment-13154938
]
[email protected] commented on OOZIE-610:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2909/
-----------------------------------------------------------
Review request for oozie.
Summary
-------
breaks the Oozie sharelib into one subdir per action type
This addresses bug OOZIE-610.
https://issues.apache.org/jira/browse/OOZIE-610
Diffs
-----
/trunk/core/pom.xml 1204834
/trunk/core/src/main/java/org/apache/oozie/action/ActionExecutor.java 1204834
/trunk/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java
1204834
/trunk/core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java
1204834
/trunk/core/src/main/java/org/apache/oozie/action/hadoop/PigActionExecutor.java
1204834
/trunk/core/src/main/java/org/apache/oozie/service/WorkflowAppService.java
1204834
/trunk/docs/src/site/twiki/DG_Examples.twiki 1204834
/trunk/docs/src/site/twiki/DG_QuickStart.twiki 1204834
/trunk/examples/pom.xml 1204834
/trunk/examples/src/main/apps/custom-main/job.properties 1204834
/trunk/examples/src/main/apps/demo/job.properties 1204834
/trunk/examples/src/main/apps/hadoop-el/job.properties 1204834
/trunk/examples/src/main/apps/java-main/job.properties 1204834
/trunk/examples/src/main/apps/pig/job.properties 1204834
/trunk/examples/src/main/apps/streaming/job.properties 1204834
/trunk/pom.xml 1204834
/trunk/sharelib/pig/pom.xml PRE-CREATION
/trunk/sharelib/pom.xml 1204834
/trunk/sharelib/streaming/pom.xml PRE-CREATION
/trunk/src/main/assemblies/distro.xml 1204834
/trunk/src/main/assemblies/examples.xml 1204834
/trunk/src/main/assemblies/partial-sharelib.xml PRE-CREATION
/trunk/src/main/assemblies/sharelib.xml 1204834
Diff: https://reviews.apache.org/r/2909/diff
Testing
-------
built, installed and run pig and streaming examples
Thanks,
Alejandro
> Oozie system share lib should have jars per action type
> -------------------------------------------------------
>
> Key: OOZIE-610
> URL: https://issues.apache.org/jira/browse/OOZIE-610
> Project: Oozie
> Issue Type: Improvement
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently Oozie share lib is a single directory with the the JARs required by
> the different action types (mapreduce-streaming & pig).
> As more action types are added to Oozie (ie Hive & Sqoop), the Oozie share
> lib will grow significantly in the number of JARs it has.
> This creates a few issues:
> * The classpath for an action grows significantly
> * For a given action only a portion of the classpath is useful, the rest is
> dead code
> * As more actions are added, the chances of conflicting dependencies grows.
> As I'm working on integrating Hive action (OOZIE-68), I'm running into the
> issue described in the #3 bullet item. Pig 0.9.0 requires antlr-runtime 3.4
> to work properly, while Hive requires antlr-runtime 3.0.1 to work properly.
> Because the current sharelib aggregates all dependencies and resolves into a
> single version of each one of them, only one version of antlr-runtime makes
> it. And because of this, either Pig or Hive works but not both.
> This JIRA proposes to add one subdirectory per action (when the action
> requires specific JARs) to the Oozie share lib, for example:
> share/lib/pig/*
> share/lib/streaming/*
> share/lib/hive/*
> share/lib/sqoop/*
> Then, the ActionExecutor for each action type will add to the action
> classpath all the JARs for the corresponding action only.
> This would move the resolution for the Oozie share lib from the
> submit-command to the action-executor.
> Note that this change will not break workflow applications using Oozie system
> share library as the change will be transparent to applications.
> Finally, the sharelib maven submodule becomes an aggregator for the sharelibs
> for each action.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
