Robert Kanter created OOZIE-2601:
------------------------------------
Summary: Ability to use local paths for the sharelib
Key: OOZIE-2601
URL: https://issues.apache.org/jira/browse/OOZIE-2601
Project: Oozie
Issue Type: New Feature
Affects Versions: oya
Reporter: Robert Kanter
Fix For: oya
With OOZIE-2590, as part of OOZIE-1770 Oozie on Yarn work, Oozie now has full
control over the classpath given to the Launcher AM. In a cluster where all
nodes have everything installed locally (in the same paths), it should be
possible to have the Launcher AM reference the local jars instead of having to
localize them from HDFS.
For example, if you have Hive installed on all nodes at {{/usr/lib/hive/}} and
all Hive jars under {{/usr/lib/hive/lib/}}, we could have the Launcher AM add
{{/usr/lib/hive/lib}} to its classpath. This saves on the overhead of
localizing the same jars from the hive sharelib in HDFS.
I think the best way to implement this is to augment the [Sharelib Mapping
File|https://oozie.apache.org/docs/4.2.0/AG_Install.html#Oozie_Share_Lib]
feature to accept {{file:///}} paths.
If we had this also work with the "oozie" sharelib and the Oozie jars in the
individual sharelibs (e.g. have the Mapping file take comma-separated
dirs/jars), then in a cluster with everything installed on all of the nodes,
you wouldn't need to bother with the sharelib at all!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
