[
https://issues.apache.org/jira/browse/OOZIE-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15347392#comment-15347392
]
Robert Kanter commented on OOZIE-2389:
--------------------------------------
At the time, that seemed to be required, but if it's no longer needed, I'm
happy if we can get rid of that. In my experience, that property generally
causes more problems than it solves. Unfortunately, I'm not sure if there's a
good way to verify that this isn't needed anymore. The spark example we ship
doesn't do very much IIRC, so it would be good to check with some more
complicated Spark apps. And with PySpark.
> Spark action failed with error starting MRAppMaster
> ---------------------------------------------------
>
> Key: OOZIE-2389
> URL: https://issues.apache.org/jira/browse/OOZIE-2389
> Project: Oozie
> Issue Type: Bug
> Affects Versions: 4.2.0
> Reporter: Hunt Tang
> Priority: Blocker
> Attachments: application_1445333963855_0050.log
>
>
> I used spark-examples-1.5.1-hadoop2.6.0.jar to generate a test spark action
> in Oozie, it succeeded sometimes, but in most of the times, it failed.
> I checked the Hadoop job history, and it said
> {quote}
> ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting
> MRAppMaster
> java.lang.NoSuchMethodError:
> org.apache.hadoop.mapred.TaskLog.createLogSyncer()Ljava/util/concurrent/ScheduledExecutorService;
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.<init>(MRAppMaster.java:244)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.<init>(MRAppMaster.java:227)
> at
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1412)
> 2015-10-22 17:01:56,203 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting
> with status 1
> {quote}
> I'm using Hadoop 2.6.0, Spark 1.5.1, Oozie 4.2.0, and I put
> spark-assembly-1.5.1-hadoop2.6.0.jar together with the example jar in the
> lib, and my workflow.xml is as followings.
> {code:xml}
> <action name="sparkTest">
> <spark xmlns="uri:oozie:spark-action:0.1">
> <job-tracker>${jobTracker}</job-tracker>
> <name-node>${nameNode}</name-node>
> <master>yarn-cluster</master>
> <name>SparkPi</name>
> <class>org.apache.spark.examples.JavaSparkPi</class>
>
> <jar>${nameNode}/sparkOozieTest/workflow/lib/spark-examples-1.5.1-hadoop2.6.0.jar</jar>
> <arg>10</arg>
> </spark>
> <ok to="end"/>
> <error to="fail"/>
> </action>
> {code}
> As it indeed succeeded sometimes, I don't think there is anything wrong with
> my configs. So is it an Oozie bug or is there anything I missed?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
