Hi again!
As I previously wrote in here (
http://mail-archives.apache.org/mod_mbox/oozie-user/201507.mbox/%3CCALBGZ8o4n27S8w6fn3HFxfzJmZbA9Gsz71Ewg%2Br6XEFCZTFpPQ%40mail.gmail.com%3E)
I'm running an oozie distro 4.2.0 built against hadoop 2.7.0.
I'm trying to execute a fat spark jar that I developed using oozie spark
action. The jar itself is perfectly running using a local/standalone/yarn
master mode via normal spark-submit.
My workflow is pretty simple
<workflow-app xmlns='uri:oozie:workflow:0.5' name='SparkBatch'>
<start to='spark-node' />
<action name='spark-node'>
<spark xmlns="uri:oozie:spark-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<master>${master}</master>
<name>sparkbatch</name>
<class>com.sparkBatch.storageArchitecture.App</class>
<jar>${nameNode}/user/${wf:user()}/${batchAppFolder}/lib/${jarName}</jar>
<!-- opts when using standalone/yarn -->
<spark-opts>--num-executors 1 --driver-memory 2g
--executor-memory 3g --executor-cores 4 --queue default</spark-opts>
</spark>
<ok to="end"/>
<error to="fail"/>
</action>
<kill name="fail">
<message>Spark Job failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
I can submit the workflow on oozie server but the job status is stuck on
RUNNING, no errors in the oozie job log.
When I open the application console on hadoop I have this error:
Application application_1437460801014_0011 failed 2 times due to AM
Container for appattempt_1437460801014_0011_000002 exited with exitCode: 1
For more detailed output, check application tracking page:
http://Matteos-MBP.local:8088/cluster/app/application_1437460801014_0011Then,
click on links to logs of each attempt.
Diagnostics: Exception from container-launch.
Container id: container_1437460801014_0011_02_000001
Exit code: 1
Stack trace: ExitCodeException exitCode=1:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
at org.apache.hadoop.util.Shell.run(Shell.java:456)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722)
at
org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
Failing this attempt. Failing the application.
and checking the yarn log via yarn logs -applicationId appId I have the
following:
2015-07-20 22:51:36,794 INFO [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
application appattempt_1437424593766_0001_000002
2015-07-20 22:51:36,938 ERROR [main]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
java.lang.NoSuchMethodError:
org.apache.hadoop.http.HttpConfig.setPolicy(Lorg/apache/hadoop/http/HttpConfig$Policy;)V
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1364)
2015-07-20 22:51:36,940 INFO [Thread-1]
org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster received a
signal. Signaling RMCommunicator and JobHistoryEventHandler.
>From the last error it seems that some component is invoking a method which
has been removed since hadoop 2.3.0 and it is not found on the current
version that I'm using (2.7.0)
P.S I can correctly execute the oozie spark example which works with a
local master
I think it's still a problem of configuration/version between oozie and
hadoop but after two days I can't figure out what is the problem. Did
somebody already face this weird error?
--
Matteo Remo Luzzi
