[
https://issues.apache.org/jira/browse/OOZIE-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14970298#comment-14970298
]
Hunt Tang commented on OOZIE-2389:
----------------------------------
[~rkanter] - As you mentioned, I checked my Oozie sharelib and found that the
spark-core jar I was using was spark-core_2.10-1.1.0.jar, however it should be
version 1.5.1. I think it's because I didn't set spark.version when I was
building Oozie. If you can help to update the Oozie building webpage, it will
be nice!
At the meanwhile, there are two tachyon jars (I have never used them) which by
default in version 0.5.0 should be replaced by version 0.6.4, otherwise, it
would encounter another exception. And I could not find any Maven property
which related to tachyon.version, so I'm not sure whether there is a way to
change its version in Oozie build phase.
Now, seems my job is working great! Thanks a lot for your kind remind!
> Spark action failed with error starting MRAppMaster
> ---------------------------------------------------
>
> Key: OOZIE-2389
> URL: https://issues.apache.org/jira/browse/OOZIE-2389
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.2.0
> Reporter: Hunt Tang
> Priority: Blocker
> Attachments: application_1445333963855_0050.log
>
>
> I used spark-examples-1.5.1-hadoop2.6.0.jar to generate a test spark action
> in Oozie, it succeeded sometimes, but in most of the times, it failed.
> I checked the Hadoop job history, and it said
> {quote}
> ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting
> MRAppMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.mapred.TaskLog.createLogSyncer()Ljava/util/concurrent/ScheduledExecutorService;
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.<init>(MRAppMaster.java:244)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.<init>(MRAppMaster.java:227)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1412)
> 2015-10-22 17:01:56,203 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting
> with status 1
> {quote}
> I'm using Hadoop 2.6.0, Spark 1.5.1, Oozie 4.2.0, and I put
> spark-assembly-1.5.1-hadoop2.6.0.jar together with the example jar in the
> lib, and my workflow.xml is as followings.
> {code:xml}
> <action name="sparkTest">
> <spark xmlns="uri:oozie:spark-action:0.1">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <master>yarn-cluster</master>
> <name>SparkPi</name>
> <class>org.apache.spark.examples.JavaSparkPi</class>
>
> <jar>${nameNode}/sparkOozieTest/workflow/lib/spark-examples-1.5.1-hadoop2.6.0.jar</jar>
> <arg>10</arg>
> </spark>
> <ok to="end"/>
> <error to="fail"/>
> </action>
> {code}
> As it indeed succeeded sometimes, I don't think there is anything wrong with
> my configs. So is it an Oozie bug or is there anything I missed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
