Did you upload the 'sharelib' correctly?
On Tuesday, August 26, 2014 8:06 AM, Charles Robertson <[email protected]> wrote: Update: I tried switching the Hive action to a Shell action and calling the Hive command. The log was less informative, but I still received error code JA018 with error message 'Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]'. This smells to me like it can't find the executables when running the job. The log shows that 'User' is root (I have submitted and run the job from an SSH console logged in as root) which should have access to the JARs - although I can run the Hive command from the console when logged in as root. Perhaps oozie is impersonating root, and not getting the same classpath or environment variables or somesuch? Regards, Charles On 26 August 2014 13:33, Charles Robertson <[email protected]> wrote: > Hi all, > > My Hive action is failing with the following error: > > Launcher exception: org/apache/hadoop/hive/conf/HiveConf > java.lang.NoClassDefFoundError: org/apache/hadoop/hive/conf/HiveConf > at > org.apache.oozie.action.hadoop.HiveMain.setUpHiveSite(HiveMain.java:182) > at org.apache.oozie.action.hadoop.HiveMain.run(HiveMain.java:196) > at > org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:38) > at org.apache.oozie.action.hadoop.HiveMain.main(HiveMain.java:66) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:225) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1594) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.hive.conf.HiveConf > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > ... 17 more > > Error Code: JA018 > Error Message: org/apache/hadoop/hive/conf/HiveConf > > This is my hive action: > <hive xmlns="uri:oozie:hive-action:0.2"> > <job-tracker>[private DNS name]:8050</job-tracker> > <name-node>hdfs://[private DNS name]:8020</name-node> > <script>score_tweets.sql</script> > </hive> > > job.properties: > nameNode=hdfs://[private DNS name]:8020 > jobTracker=[private DNS name]:8050 > queueName=default > oozie.libpath=${nameNode}/user/oozie/share/lib > oozie.wf.application.path=${nameNode}/user/root/oozie > > I am using oozie 4.0.0 on HDP 2.1 > > Googling produced some useful things (see > https://support.pivotal.io/hc/en-us/articles/202563453-Oozie-hive-action-fails-with-java-lang-NoClassDefFoundError-org-apache-hadoop-hive-conf-HiveConf- > and > http://stackoverflow.com/questions/18369605/error-while-running-hive-action-in-oozie), > but I've tried these, and I'm still getting the same error. > > I have: > - ensured that hive-exec is in /user/oozie/share/lib (also in > /user/root/share/lib as this job is run as root) > - ensured that hive-exec exists in a path in the common.loader property in > the catalina.properties file > - even put the 'oozie.service.WorkflowAppService.system.libpath' property > into hive-site.xml like the stackoverflow answer suggested, even though > this doesn't make much sense to me (it is in oozie-site.xml and I have > overriden it to point hard-coded to /user/oozie/share/lib) > - not done anything with hive-default.xml since the documentation says > this is now ignored. > > Does anyone have any suggestions? > > Thanks, > Charles >
