[
https://issues.apache.org/jira/browse/OOZIE-2601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295801#comment-16295801
]
Rohini Palaniswamy commented on OOZIE-2601:
-------------------------------------------
bq. We should take care of the order the user specifies their dependencies
I am not sure you can do much there. Yarn just does $PWD/* to include all the
localized files. You can either add the local file directories to the end or to
the beginning. I would suggest end so that any user jars in $PWD take
precedence.
> Ability to use local paths for the sharelib
> -------------------------------------------
>
> Key: OOZIE-2601
> URL: https://issues.apache.org/jira/browse/OOZIE-2601
> Project: Oozie
> Issue Type: New Feature
> Affects Versions: oya
> Reporter: Robert Kanter
> Assignee: Attila Sasvari
> Fix For: oya, 5.0.0b1
>
> Attachments: OOZIE-2601-01.patch, OOZIE-2601-03.patch,
> OOZIE-2601-05.patch, OOZIE-2601-06.patch, OOZIE-2601-07.patch,
> OOZIE-2601-09.patch, OOZIE-2601-10.patch, OOZIE-2601-11.patch
>
>
> With OOZIE-2590, as part of OOZIE-1770 Oozie on Yarn work, Oozie now has full
> control over the classpath given to the Launcher AM. In a cluster where all
> nodes have everything installed locally (in the same paths), it should be
> possible to have the Launcher AM reference the local jars instead of having
> to localize them from HDFS.
> For example, if you have Hive installed on all nodes at {{/usr/lib/hive/}}
> and all Hive jars under {{/usr/lib/hive/lib/}}, we could have the Launcher AM
> add {{/usr/lib/hive/lib}} to its classpath. This saves on the overhead of
> localizing the same jars from the hive sharelib in HDFS.
> I think the best way to implement this is to augment the [Sharelib Mapping
> File|https://oozie.apache.org/docs/4.2.0/AG_Install.html#Oozie_Share_Lib]
> feature to accept {{file:///}} paths.
> If we had this also work with the "oozie" sharelib and the Oozie jars in the
> individual sharelibs (e.g. have the Mapping file take comma-separated
> dirs/jars), then in a cluster with everything installed on all of the nodes,
> you wouldn't need to bother with the sharelib at all!
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
