[
https://issues.apache.org/jira/browse/OOZIE-610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154946#comment-13154946
]
Alejandro Abdelnur commented on OOZIE-610:
------------------------------------------
Forgot to mention the different between this JIRA & OOZIE-552.
OOZIE-552 allows a user to specify multiple directories in the sharelib path (a
la classpath), thus aggregating all JARs from those directories.
This JIRA selects the directory to add to the classpath based on the action
type. When running streaming action only streaming JARs are added, when running
pig action only pig JARs are added.
This JIRA only affects Oozie system sharelib, the functionality introduced by
OOZIE-552 is still available. Furthermore they are complimentary.
Note that for the case of conflicting dependencies between Pig and Hive
OOZIE-552 does not provide a solution as the the multiple directories paths are
aggregated to all actions
> Oozie system share lib should have jars per action type
> -------------------------------------------------------
>
> Key: OOZIE-610
> URL: https://issues.apache.org/jira/browse/OOZIE-610
> Project: Oozie
> Issue Type: Improvement
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
>
> Currently Oozie share lib is a single directory with the the JARs required by
> the different action types (mapreduce-streaming & pig).
> As more action types are added to Oozie (ie Hive & Sqoop), the Oozie share
> lib will grow significantly in the number of JARs it has.
> This creates a few issues:
> * The classpath for an action grows significantly
> * For a given action only a portion of the classpath is useful, the rest is
> dead code
> * As more actions are added, the chances of conflicting dependencies grows.
> As I'm working on integrating Hive action (OOZIE-68), I'm running into the
> issue described in the #3 bullet item. Pig 0.9.0 requires antlr-runtime 3.4
> to work properly, while Hive requires antlr-runtime 3.0.1 to work properly.
> Because the current sharelib aggregates all dependencies and resolves into a
> single version of each one of them, only one version of antlr-runtime makes
> it. And because of this, either Pig or Hive works but not both.
> This JIRA proposes to add one subdirectory per action (when the action
> requires specific JARs) to the Oozie share lib, for example:
> share/lib/pig/*
> share/lib/streaming/*
> share/lib/hive/*
> share/lib/sqoop/*
> Then, the ActionExecutor for each action type will add to the action
> classpath all the JARs for the corresponding action only.
> This would move the resolution for the Oozie share lib from the
> submit-command to the action-executor.
> Note that this change will not break workflow applications using Oozie system
> share library as the change will be transparent to applications.
> Finally, the sharelib maven submodule becomes an aggregator for the sharelibs
> for each action.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
