Bowen is correct. You need to upload using oozie-setup.sh sharelib create
to sharelib. It copies sharelib to a new timestamp directory. One can
upload multiple version of sharelib. Server will cleanup the older
sharelib. Earlier user have to clean it manually.
If you don¹t want to use oozie-setup.sh to setup sharelib, you can use
metafile configuration.
For Doc..
------------
Sharelib mapping file can be also configured. Configured file is a key
value mapping, where key will be the sharelib name for the
action and value is a comma separated list of DFS directories or jar files.
This can be configured in oozie-site.xml as :
<verbatim>
<!-- OOZIE -->
<property>
<name>oozie.service.ShareLibService.mapping.file</name>
<value></value>
<description>
Sharelib mapping files contains list of key=value,
where key will be the sharelib name for the action and value
is a comma separated list of
DFS directories or jar files.
Example.
oozie.pig_10=hdfs:///share/lib/pig/pig-0.10.1/lib/
oozie.pig=hdfs:///share/lib/pig/pig-0.11.1/lib/
oozie.distcp=hdfs:///share/lib/hadoop-2.2.0/share/hadoop/tools/lib/hadoop-d
istcp-2.2.0.jar
</description>
</property>
</verbatim>
------------
Puru.
On 4/9/14, 12:48 PM, "bowen zhang" <[email protected]> wrote:
>no, it's not. Before, user can manually upload sharelib to hdfs by using
>"hadoop dfs -put share share" command. Right now, you have to use the
>³uoozie-setup.sh sharelib create" command to upload sharelib. Puru from
>yahoo can address this issue.
>
>
>On Wednesday, April 9, 2014 12:10 PM, Satish Mittal
><[email protected]> wrote:
>
>Hi,
>
>I built oozie trunk and started oozie server with default config options.
>The oozie share lib directory is present in HDFS at usual location
>/user/oozie/share/lib. However oozie server logs contained:
>
>2014-04-09 18:17:57,507 WARN main ConfigurationService - SERVER[dev1]
>USER[-] GROUP[-] Configuration property
>[oozie.service.ShareLibService.mapping.file] not found, using default []
>2014-04-09 18:29:05,118 INFO main ShareLibService - SERVER[dev1] USER[-]
>GROUP[-] *No share lib directory found*
>2014-04-09 18:31:10,068 ERROR main ShareLibService - SERVER[dev1] USER[-]
>GROUP[-] Not able to cache shareLib. Admin need to issue oozlie cli
>command
>to update sharelib.
>java.lang.NullPointerException
> at
>org.apache.oozie.service.ShareLibService.init(ShareLibService.java:108)
> at
>org.apache.oozie.service.Services.setServiceInternal(Services.java:368)
> at org.apache.oozie.service.Services.setService(Services.java:354)
> at
>org.apache.oozie.service.Services.loadServices(Services.java:287)
> at org.apache.oozie.service.Services.init(Services.java:208)
> at
>org.apache.oozie.servlet.ServicesLoader.contextInitialized(ServicesLoader.
>java:45)
> at
>org.apache.catalina.core.StandardContext.listenerStart(StandardContext.jav
>a:4206)
> at
>org.apache.catalina.core.StandardContext.start(StandardContext.java:4705)
> at
>org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java
>:799)
> at
>org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:779)
> at
>org.apache.catalina.core.StandardHost.addChild(StandardHost.java:601)
>at
>org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:67
>5)
> at
>org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:6
>01)
> at
>org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:502)
> at
>org.apache.catalina.startup.HostConfig.start(HostConfig.java:1317)
> at
>org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:324)
> at
>org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupp
>ort.java:142)
> at
>org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1065)
>at org.apache.catalina.core.StandardHost.start(StandardHost.java:840)
> at
>org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1057)
>at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
> at
>org.apache.catalina.core.StandardService.start(StandardService.java:525)
>at org.apache.catalina.core.StandardServer.start(StandardServer.java:754)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
>sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
>57)
> at
>sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorIm
>pl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
>2014-04-09 18:31:15,116 INFO main Services - SERVER[dev1] Initialized
>2014-04-09 18:31:15,124 INFO main Services - SERVER[dev1] Running with
>JARs for Hadoop version [0.20.2-cdh3u5]
>2014-04-09 18:31:15,125 INFO main Services - SERVER[dev1] Oozie System ID
>[oozie-oozi] started!
>
>Consequently workflow action launch submit failed since it couldn't find
>oozie share lib location:
>
>JOB[0000002-140409110726107-oozie-oozi-W]
>ACTION[0000002-140409110726107-oozie-oozi-W@eviction] Error starting
>action
>[eviction]. ErrorType [ERROR], ErrorCode [IndexOutOfBoundsException],
>Message [IndexOutOfBoundsException: Index: 0, Size: 0]
>org.apache.oozie.action.ActionExecutorException:
>IndexOutOfBoundsException:
>Index: 0, Size: 0
>at
>org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.jav
>a:401)
>at
>org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActio
>nExecutor.java:929)
>at
>org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor
>.java:1084)
>at
>org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXComman
>d.java:228)
>at
>org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXComman
>d.java:63)
>at org.apache.oozie.command.XCommand.call(XCommand.java:283)
>at
>org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Calla
>bleQueueService.java:323)
>at
>org.apache.oozie.service.CallableQueueService$CompositeCallable.call(Calla
>bleQueueService.java:252)
>at
>org.apache.oozie.service.CallableQueueService$CallableWrapper.run(Callable
>QueueService.java:174)
>at
>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:
>1110)
>at
>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java
>:603)
>at java.lang.Thread.run(Thread.java:679)
>Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>at java.util.ArrayList.rangeCheck(ArrayList.java:571)
>at java.util.ArrayList.get(ArrayList.java:349)
>at
>org.apache.oozie.action.hadoop.JavaActionExecutor.addSystemShareLibForActi
>on(JavaActionExecutor.java:557)
>at
>org.apache.oozie.action.hadoop.JavaActionExecutor.addAllShareLibs(JavaActi
>onExecutor.java:649)
>at
>org.apache.oozie.action.hadoop.JavaActionExecutor.setLibFilesArchives(Java
>ActionExecutor.java:640)
>at
>org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActio
>nExecutor.java:834)
>... 10 more
>
>Does this mean that *current oozie trunk is not backward compatible with
>earlier oozie behavior*, where I could just deploy oozie share lib in HDFS
>at the system libpath provided in config, and oozie server would then find
>it and work with it?
>
>Regards,
>Satish
>
>--
>_____________________________________________________________
>The information contained in this communication is intended solely for
>the
>use of the individual or entity to whom it is addressed and others
>authorized to receive it. It may contain confidential or legally
>privileged
>information. If you are not the intended recipient you are hereby
>notified
>that any disclosure, copying, distribution or taking any action in
>reliance
>on the contents of this information is strictly prohibited and may be
>unlawful. If you have received this communication in error, please notify
>us immediately by responding to this email and then delete it from your
>system. The firm is neither liable for the proper and complete
>transmission
>of the information contained in this communication nor for any delay in
>its
>receipt.
