Hello, I tried a new installation procedure for shreLib installation after overcoming issues like: user oozie cannot impersonate user oozie etc. I got share/libs sucessfully installed to HDFS under /user/oozie/share/lib/lib_20141107094835 which I guess is fine. BUT the result of the command oozie admin -shareliblist
returns null The commad for sharelib creation doesn't print any error to output nor oozie servers logs are free of any error. Can somebody help me to nail it down? Where oozie keep the information which share lib should be used? Thanks for helping me out Jakub On 4 February 2015 at 23:27, Jakub Stransky <[email protected]> wrote: > No, I didn't I used instruction from apache oozie sites 4.1.0 which are > not different from the old one. WIll give a try! > > Thanks for pointing this out > Jakub > > > On 4 February 2015 at 22:55, Mohammad Islam <[email protected]> > wrote: > >> Hi Jakub,Shared library structure was changed in 4.1.0. >> It is now in /user/oozie/shared/lib/lib_$timestamp. >> Did you follow the new instructions?Some discussion at:How-to: Use the >> ShareLib in Apache Oozie (CDH 5) >> >> | | >> | | | | | | >> | How-to: Use the ShareLib in Apache Oozie (CDH 5)drwxr-xr-x >> share/lib/lib_20140403151601/distcp -rw-r--r-- >> share/lib/lib_20140403151601/distcp/hadoop-distcp-2.3.0-cdh5.0.0.jar | >> | | >> | View on blog.cloudera.com | Preview by Yahoo | >> | | >> | | >> >> >> Regards,Mohamamd >> >> >> >> >> On Wednesday, February 4, 2015 7:03 AM, Jakub Stransky < >> [email protected]> wrote: >> >> >> Hello experienced oozie users, >> >> we have upgraded our infrastructure from hdp 2.0 to hdp 2.2 which means >> upgrade from hadoop 2.2.0 to 2.6.0 and oozie from 4.0.0 to 4.1.0. We are >> using simple authentication etc. When the job gets submitted oozie >> launcher >> is failing with following: >> >> 2015-02-04 09:56:33,364 WARN [main] >> org.apache.hadoop.mapred.YarnChild: Exception running child : >> java.io.IOException: Split class >> org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not >> found >> at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363) >> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:423) >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) >> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) >> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) >> Caused by: java.lang.ClassNotFoundException: Class >> org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not >> found >> at >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1982) >> at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361) >> ... 7 more >> >> 2015-02-04 09:56:33,373 INFO [main] org.apache.hadoop.mapred.Task: >> Runnning cleanup for the task >> >> Found that this classes are contained in oozie share/lib uploaded to HDFS >> to /user/${user.name}/share/lib according to oozie-site.xml >> configuration: >> >> <property> >> <name>oozie.service.WorkflowAppService.system.libpath</name> >> <value>/user/${user.name}/share/lib</value> >> </property> >> >> where ${user.name} is user under which oozie server is running e.g. >> oozie. >> For some reason those libs doesn't get propagated to >> mapreduce.application.classpath and hence ClassNotFoundException. >> >> Tried to change following property to true: >> >> <property> >> <name>use.system.libpath.for.mapreduce.and.pig.jobs</name> >> <value>true</value> >> </property> >> >> But with no difference. I am not sure what I am missing in installation >> instruction here >> < >> http://oozie.apache.org/docs/4.1.0/DG_QuickStart.html#Oozie_Share_Lib_Installation >> > >> >> Any hint would be appreciated. >> >> Thx >> >> >> >> > > > > -- > Jakub Stransky > cz.linkedin.com/in/jakubstransky > > -- Jakub Stransky cz.linkedin.com/in/jakubstransky
