Hi, I also faced similar a problem while trying to deploy spark applications through the spark action. I switched to the shell action as a workaround. You can follow the link below for a more descriptive answer.
http://stackoverflow.com/questions/31450828/oozie-job-wont-run-if-using-pyspark-in-sparkaction/33094709#33094709 I am interested however to know how to achieve this through the spark action. Hoping that someone can site a good example. Regards, Nitin Kumar On 14-Jun-2016 9:35 PM, "Pierre Villard" <[email protected]> wrote: > Hi, > > I am trying to run a Spark action (Oozie 4.2.0) and everything works fine > if my action is run in local mode. As soon as I switch to cluster mode with > yarn as master I get a NullPointerException from Oozie: > > JOB[0000724-160602113052690-oozie-oozi-W] > ACTION[0000724-160602113052690-oozie-oozi-W@submit-spark-job] Error > starting action [submit-spark-job]. ErrorType [ERROR], ErrorCode > [NullPointerException], Message [NullPointerException: null] > org.apache.oozie.action.ActionExecutorException: NullPointerException: null > at > > org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:445) > at > > org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1139) > at > > org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1293) > at > > org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:250) > at > > org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:64) > at org.apache.oozie.command.XCommand.call(XCommand.java:286) > at org.apache.oozie.command.XCommand.call(XCommand.java:356) > at > org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:434) > at > org.apache.oozie.command.wf.SignalXCommand.execute(SignalXCommand.java:76) > at org.apache.oozie.command.XCommand.call(XCommand.java:286) > at org.apache.oozie.command.XCommand.call(XCommand.java:356) > at > > org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:280) > at > > org.apache.oozie.command.wf.ActionEndXCommand.execute(ActionEndXCommand.java:61) > at org.apache.oozie.command.XCommand.call(XCommand.java:286) > at org.apache.oozie.command.XCommand.call(XCommand.java:356) > at > > org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:244) > at > > org.apache.oozie.command.wf.ActionCheckXCommand.execute(ActionCheckXCommand.java:56) > at org.apache.oozie.command.XCommand.call(XCommand.java:286) > at > > org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175) > at > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.NullPointerException > at > > org.apache.oozie.action.hadoop.SparkActionExecutor.setupActionConf(SparkActionExecutor.java:79) > at > > org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1042) > ... 20 more > > I am wondering if this is somehow related to: > https://issues.apache.org/jira/browse/SPARK-7706 > > In any case, what is the cause? is there a workaround? > > Thanks for your help, > Pierre >
