Attila Sasvari updated OOZIE-2601:
    Attachment: OOZIE-2601-09.patch

patch rebase + addressed review comments

> Ability to use local paths for the sharelib
> -------------------------------------------
>                 Key: OOZIE-2601
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2601
>             Project: Oozie
>          Issue Type: New Feature
>    Affects Versions: oya
>            Reporter: Robert Kanter
>            Assignee: Attila Sasvari
>             Fix For: oya
>         Attachments: OOZIE-2601-01.patch, OOZIE-2601-03.patch, 
> OOZIE-2601-05.patch, OOZIE-2601-06.patch, OOZIE-2601-07.patch, 
> OOZIE-2601-09.patch
> With OOZIE-2590, as part of OOZIE-1770 Oozie on Yarn work, Oozie now has full 
> control over the classpath given to the Launcher AM.  In a cluster where all 
> nodes have everything installed locally (in the same paths), it should be 
> possible to have the Launcher AM reference the local jars instead of having 
> to localize them from HDFS.
> For example, if you have Hive installed on all nodes at {{/usr/lib/hive/}} 
> and all Hive jars under {{/usr/lib/hive/lib/}}, we could have the Launcher AM 
> add {{/usr/lib/hive/lib}} to its classpath.  This saves on the overhead of 
> localizing the same jars from the hive sharelib in HDFS.  
> I think the best way to implement this is to augment the [Sharelib Mapping 
> File|https://oozie.apache.org/docs/4.2.0/AG_Install.html#Oozie_Share_Lib] 
> feature to accept {{file:///}} paths.
> If we had this also work with the "oozie" sharelib and the Oozie jars in the 
> individual sharelibs (e.g. have the Mapping file take comma-separated 
> dirs/jars), then in a cluster with everything installed on all of the nodes, 
> you wouldn't need to bother with the sharelib at all!

This message was sent by Atlassian JIRA

Reply via email to