Hi, Oozie 4.2 does not support Spark 2, but the error message points to the sharelib is not being installed properly. Could you provide the output of "oozie admin -shareliblist spark" command?
The upcoming Oozie 4.3 will support Spark 2 given it was compiled with the spark2 profile. You may find an RC sent out to the developer list just a few days ago: http://mail-archives.apache.org/mod_mbox/oozie-dev/201610.mbox/%3CCAHZHDpBrhJrAn5KTAbxtdWGnQkm6TFZfJNCpwtrqeu_Zs9xf7w%40mail.gmail.com%3E It's not a final release yet, so I wouldn't recommend to use it in production, but if you have the time to test it out, we would appreciate the feedback about it. gp On Fri, Oct 28, 2016 at 3:32 AM, lk_hadoop <[email protected]> wrote: > hi,all: > when Ｉ user oozie 4.2 spark action with spark2.0 ,I got error: > > 2016-10-28 09:26:49,167 WARN SparkActionExecutor:523 - SERVER[ > dev0.youedata.com] USER[hadoop] GROUP[-] TOKEN[] APP[SparkParquet] > JOB[0000013-161026142257915-oozie-hado-W] ACTION[0000013- > 161026142257915-oozie-hado-W@spark-node] Launcher exception: > java.lang.ClassNotFoundException: > Class org.apache.oozie.action.hadoop.SparkMain not found > java.lang.RuntimeException: java.lang.ClassNotFoundException: Class > org.apache.oozie.action.hadoop.SparkMain not found > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2195) > at org.apache.oozie.action.hadoop.LauncherMapper.map( > LauncherMapper.java:234) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at org.apache.hadoop.mapred.LocalContainerLauncher$ > EventHandler.runSubtask(LocalContainerLauncher.java:380) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask( > LocalContainerLauncher.java:301) > at org.apache.hadoop.mapred.LocalContainerLauncher$ > EventHandler.access$200(LocalContainerLauncher.java:187) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run( > LocalContainerLauncher.java:230) > at java.util.concurrent.Executors$RunnableAdapter. > call(Executors.java:511) > at java.util.concurrent.FutureTask.run(FutureTask.java:266) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.ClassNotFoundException: Class > org.apache.oozie.action.hadoop.SparkMain not found > at org.apache.hadoop.conf.Configuration.getClassByName( > Configuration.java:2101) > at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2193) > ... 13 more > > > 2016-10-28 > > > lk_hadoop -- Peter Cseh Software Engineer <http://www.cloudera.com>
