Hello experienced oozie users,
we have upgraded our infrastructure from hdp 2.0 to hdp 2.2 which means
upgrade from hadoop 2.2.0 to 2.6.0 and oozie from 4.0.0 to 4.1.0. We are
using simple authentication etc. When the job gets submitted oozie launcher
is failing with following:
2015-02-04 09:56:33,364 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.io.IOException: Split class
org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not
found
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:363)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:423)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.ClassNotFoundException: Class
org.apache.oozie.action.hadoop.OozieLauncherInputFormat$EmptySplit not
found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1982)
at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:361)
... 7 more
2015-02-04 09:56:33,373 INFO [main] org.apache.hadoop.mapred.Task:
Runnning cleanup for the task
Found that this classes are contained in oozie share/lib uploaded to HDFS
to /user/${user.name}/share/lib according to oozie-site.xml configuration:
<property>
<name>oozie.service.WorkflowAppService.system.libpath</name>
<value>/user/${user.name}/share/lib</value>
</property>
where ${user.name} is user under which oozie server is running e.g. oozie.
For some reason those libs doesn't get propagated to
mapreduce.application.classpath and hence ClassNotFoundException.
Tried to change following property to true:
<property>
<name>use.system.libpath.for.mapreduce.and.pig.jobs</name>
<value>true</value>
</property>
But with no difference. I am not sure what I am missing in installation
instruction here
<http://oozie.apache.org/docs/4.1.0/DG_QuickStart.html#Oozie_Share_Lib_Installation>
Any hint would be appreciated.
Thx
