[
https://issues.apache.org/jira/browse/OOZIE-1584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920926#comment-13920926
]
Josh Spiegel commented on OOZIE-1584:
-------------------------------------
[~rohini],
I am using hadoop-2.2.0 (CDH5 beta 2) MR1 and I don't think it creates
symlinks in the working directory when there isn't a uri fragment.
In my case, I was adding native libraries (.so) files to the sharelib for a
custom extension. I can work around the new behavior but I raised the issue on
this bug because I suspect the change in behavior is *probably* unintentional.
The current behavior (I think) is that non-jar files get added to the cache and
added to the classpath but do not get symlinked in the working directory. The
old behavior was that they get added to the cache and symlinked but are not
added to the classpath.
Searching the sharelib recursively doesn't impact me.
> Setup sharelib using script and pickup latest(honor ship.launcher) and remove
> DFS dependency at startup.
> --------------------------------------------------------------------------------------------------------
>
> Key: OOZIE-1584
> URL: https://issues.apache.org/jira/browse/OOZIE-1584
> Project: Oozie
> Issue Type: Sub-task
> Reporter: purshotam shah
> Assignee: purshotam shah
> Fix For: trunk
>
> Attachments: OOZIE-1584_v2.patch, OOZIE-1584_v3.patch,
> OOZIE-1584_v4.patch, OOZIE-1584_v5.patch, OOZIE-1584_v6.patch
>
>
> Supported features.
> 1. If Oozie.action.ship.launcher.jar = true then, Oozie auto-deploy jar at
> server starts.
> 2. Admin copies sharelib to new timestamp directory(using sharelibCLI) and
> issues invalidate command(using oozieCLI) to Oozie server. Oozie picks latest
> shared lib.
> 3.Oozie server doesn't auto deploy sharelib at startup. It picks the latest
> sharelib jar pushed by admin(based on timestamp).
> DFS after generating launcher jar by ooozie server and sharelib by cli
> -------------------
> makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls
> /user/purushah/share/lib/
> /user/purushah/share/lib/launcher_20131017092254
> /user/purushah/share/lib/launcher_20131017093814
> /user/purushah/share/lib/launcher_20131017094652
> /user/purushah/share/lib/launcher_20131017094836
> /user/purushah/share/lib/launcher_20131017095549
> /user/purushah/share/lib/lib_20131017092806
> makebag-lm:example purushah$
> Purging.
> -------------
> There are two set( launcher_ for launcher jars and Lib_ for shared lib) of
> directory and purging happens for both at startup.
> Anything older than ( defined in property) is purged.
> We always keep 2 set of directory, purging happens if number of sharelib
> directory > 2.
> Multiple version.
> -------------------
> To support multiple version, sharelib will be in below format.
> Sharelib will cache all versions and return list of dfs files for a particular
> version.
> makebag-lm:example purushah$ /var/hadoop-1.2.1/bin/hadoop fs -ls
> /user/purushah/share/lib/
> Found 15 items
> /user/purushah/share/lib/launcher_20131017092254
> /user/purushah/share/lib/launcher_20131017093814
> /user/purushah/share/lib/launcher_20131017094652
> /user/purushah/share/lib/launcher_20131017094836
> /user/purushah/share/lib/launcher_20131017095549
> /user/purushah/share/lib/lib_20131017092806/pig/...
> /user/purushah/share/lib/lib_20131017092806/pig_9/...
> /user/purushah/share/lib/lib_20131017092806/pig_10/...
> makebag-lm:example purushah$
--
This message was sent by Atlassian JIRA
(v6.2#6252)
