[
https://issues.apache.org/jira/browse/OOZIE-1584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13820344#comment-13820344
]
Rohini Palaniswamy commented on OOZIE-1584:
-------------------------------------------
We want the file to be reprocessed by the admin sharelib update command. Making
it a configuration value will require changing oozie settings. Also we have
multiple versions of pig, hive, hbase, hcat, distcp, streaming, etc deployed at
the same time. Specifying them as a single configuration grows unwieldy. So
making it a separate property file and the file can be on hdfs or oozie server.
> Setup sharelib using script and pickup latest(honor ship.launcher) and remove
> DFS dependency at startup.
> --------------------------------------------------------------------------------------------------------
>
> Key: OOZIE-1584
> URL: https://issues.apache.org/jira/browse/OOZIE-1584
> Project: Oozie
> Issue Type: Bug
> Reporter: purshotam shah
> Assignee: purshotam shah
>
> Supported features.
> 1. If Oozie.action.ship.launcher.jar = true then, Oozie auto-deploy jar at
> server starts.
> 2. Admin copies sharelib to new timestamp directory(using sharelibCLI) and
> issues invalidate command(using oozieCLI) to Oozie server. Oozie picks latest
> shared lib.
> 3.Oozie server doesn't auto deploy sharelib at startup. It picks the latest
> sharelib jar pushed by admin(based on timestamp).
> DFS after generating launcher jar by ooozie server and sharelib by cli
> -------------------
> makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls
> /user/purushah/share/lib/
> /user/purushah/share/lib/launcher_20131017092254
> /user/purushah/share/lib/launcher_20131017093814
> /user/purushah/share/lib/launcher_20131017094652
> /user/purushah/share/lib/launcher_20131017094836
> /user/purushah/share/lib/launcher_20131017095549
> /user/purushah/share/lib/lib_20131017092806
> makebag-lm:example purushah$
> Purging.
> -------------
> There are two set( launcher_ for launcher jars and Lib_ for shared lib) of
> directory and purging happens for both at startup.
> Anything older than ( defined in property) is purged.
> We always keep 2 set of directory, purging happens if number of sharelib
> directory > 2.
> Multiple version.
> -------------------
> To support multiple version, sharelib will be in below format.
> Sharelib will cache all versions and return list of dfs files for a particular
> version.
> makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls
> /user/purushah/share/lib/
> Found 15 items
> /user/purushah/share/lib/launcher_20131017092254
> /user/purushah/share/lib/launcher_20131017093814
> /user/purushah/share/lib/launcher_20131017094652
> /user/purushah/share/lib/launcher_20131017094836
> /user/purushah/share/lib/launcher_20131017095549
> /user/purushah/share/lib/lib_20131017092806/pig/...
> /user/purushah/share/lib/lib_20131017092806/pig_9/...
> /user/purushah/share/lib/lib_20131017092806/pig_10/...
> makebag-lm:example purushah$
--
This message was sent by Atlassian JIRA
(v6.1#6144)
