Hello experienced users, anybody faced similar issue? I am stack on this for 3 day and still not found what might be a problem. Since the installation of share libs is correct as I see a corect folder with a timestamp but cannot figure it out why this is not added to classpath even when I specify oozie.use.system.libpath=true in job.properties. When tried strace oozie admin -shareliblist command I can see a failing child process but can not nail it down.
Does anybody has some suggestion what to try or check? On 6 February 2015 at 10:44, Jakub Stransky <[email protected]> wrote: > Hello, > > I tried a new installation procedure for shreLib installation after > overcoming issues like: user oozie cannot impersonate user oozie etc. I got > share/libs sucessfully installed to HDFS under > /user/oozie/share/lib/lib_20141107094835 which I guess is fine. BUT the > result of the command > oozie admin -shareliblist > > returns null > > The commad for sharelib creation doesn't print any error to output nor > oozie servers logs are free of any error. Can somebody help me to nail it > down? Where oozie keep the information which share lib should be used? > > Thanks for helping me out > > Jakub > > > On 4 February 2015 at 23:27, Jakub Stransky <[email protected]> wrote: > >> No, I didn't I used instruction from apache oozie sites 4.1.0 which are >> not different from the old one. WIll give a try! >> >> Thanks for pointing this out >> Jakub >> >> >> On 4 February 2015 at 22:55, Mohammad Islam <[email protected]> >> wrote: >> >>> Hi Jakub,Shared library structure was changed in 4.1.0. >>> It is now in /user/oozie/shared/lib/lib_$timestamp. >>> Did you follow the new instructions?Some discussion at:How-to: Use the >>> ShareLib in Apache Oozie (CDH 5) >>> >>> | | >>> | | | | | | >>> | How-to: Use the ShareLib in Apache Oozie (CDH 5)drwxr-xr-x >>> share/lib/lib_20140403151601/distcp -rw-r--r-- >>> share/lib/lib_20140403151601/distcp/hadoop-distcp-2.3.0-cdh5.0.0.jar | >>> | | >>> | View on blog.cloudera.com | Preview by Yahoo | >>> | | >>> | | >>> >>> >>> Regards,Mohamamd >>> >>> >>> >>> >>> On Wednesday, February 4, 2015 7:03 AM, Jakub Stransky < >>> [email protected]> wrote: >>> >>> >>> Hello experienced oozie users, >>> >>> we have upgraded our infrastructure from hdp 2.0 to hdp 2.2 which means >>> upgrade from hadoop 2.2.0 to 2.6.0 and oozie from 4.0.0 to 4.1.0. We are >>> using simple authentication etc. When the job gets submitted oozie >>> launcher >>> is failing with following: >>> >>> 2015-02-04 09:56:33,364 WARN [main] >>> org.apache.hadoop.mapred.YarnChild: Exception running child : >>> java.io.IOException: Split class >>> org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not >>> found >>> at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363) >>> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:423) >>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) >>> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:415) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) >>> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) >>> Caused by: java.lang.ClassNotFoundException: Class >>> org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not >>> found >>> at >>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1982) >>> at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361) >>> ... 7 more >>> >>> 2015-02-04 09:56:33,373 INFO [main] org.apache.hadoop.mapred.Task: >>> Runnning cleanup for the task >>> >>> Found that this classes are contained in oozie share/lib uploaded to HDFS >>> to /user/${user.name}/share/lib according to oozie-site.xml >>> configuration: >>> >>> <property> >>> <name>oozie.service.WorkflowAppService.system.libpath</name> >>> <value>/user/${user.name}/share/lib</value> >>> </property> >>> >>> where ${user.name} is user under which oozie server is running e.g. >>> oozie. >>> For some reason those libs doesn't get propagated to >>> mapreduce.application.classpath and hence ClassNotFoundException. >>> >>> Tried to change following property to true: >>> >>> <property> >>> <name>use.system.libpath.for.mapreduce.and.pig.jobs</name> >>> <value>true</value> >>> </property> >>> >>> But with no difference. I am not sure what I am missing in installation >>> instruction here >>> < >>> http://oozie.apache.org/docs/4.1.0/DG_QuickStart.html#Oozie_Share_Lib_Installation >>> > >>> >>> Any hint would be appreciated. >>> >>> Thx >>> >>> >>> >>> >> >> >> >> -- >> Jakub Stransky >> cz.linkedin.com/in/jakubstransky >> >> > > > -- > Jakub Stransky > cz.linkedin.com/in/jakubstransky > > -- Jakub Stransky cz.linkedin.com/in/jakubstransky
