[
https://issues.apache.org/jira/browse/SPARK-5074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Or reopened SPARK-5074:
------------------------------
Reopening because this is still flaky:
https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.0,label=centos/1771/
https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.0,label=centos/1770/
https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.0,label=centos/1761/
https://amplab.cs.berkeley.edu/jenkins/job/Spark-Master-SBT/AMPLAB_JENKINS_BUILD_PROFILE=hadoop2.0,label=centos/1754/
{code}
"...e to stage failure: [Task serialization failed:
java.lang.IllegalStateException: Cannot call methods on a stopped SparkContext
org.apache.spark.SparkContext.assertNotStopped(SparkContext.scala:101)
org.apache.spark.SparkContext.broadcast(SparkContext.scala:1037)
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitMissingTasks(DAGScheduler.scala:839)
org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:778)
org.apache.spark.scheduler.DAGScheduler$$anonfun$resubmitFailedStages$3.apply(DAGScheduler.scala:590)
org.apache.spark.scheduler.DAGScheduler$$anonfun$resubmitFailedStages$3.apply(DAGScheduler.scala:589)
scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
org.apache.spark.scheduler.DAGScheduler.resubmitFailedStages(DAGScheduler.scala:589)
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1396)
org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1354)
org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48) ]" did not
equal "...e to stage failure: [Exception failure in map stage]"
{code}
> Flaky test: o.a.s.scheduler.DAGSchedulerSuite
> ---------------------------------------------
>
> Key: SPARK-5074
> URL: https://issues.apache.org/jira/browse/SPARK-5074
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Reporter: Shixiong Zhu
> Assignee: Shixiong Zhu
> Priority: Critical
> Labels: flaky-test
> Fix For: 1.3.0
>
>
> fix the following non-deterministic test in
> org.apache.spark.scheduler.DAGSchedulerSuite
> {noformat}
> [info] DAGSchedulerSuite:
> [info] - [SPARK-3353] parent stage should have lower stage id *** FAILED ***
> (27 milliseconds)
> [info] 1 did not equal 2 (DAGSchedulerSuite.scala:242)
> [info] org.scalatest.exceptions.TestFailedException:
> [info] at
> org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:500)
> [info] at
> org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1555)
> [info] at
> org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:466)
> [info] at
> org.apache.spark.scheduler.DAGSchedulerSuite$$anonfun$2.apply$mcV$sp(DAGSchedulerSuite.scala:242)
> [info] at
> org.apache.spark.scheduler.DAGSchedulerSuite$$anonfun$2.apply(DAGSchedulerSuite.scala:239)
> [info] at
> org.apache.spark.scheduler.DAGSchedulerSuite$$anonfun$2.apply(DAGSchedulerSuite.scala:239)
> [info] at
> org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
> [info] at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
> [info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
> [info] at org.scalatest.Transformer.apply(Transformer.scala:22)
> [info] at org.scalatest.Transformer.apply(Transformer.scala:20)
> [info] at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
> [info] at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
> [info] at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
> [info] at
> org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
> [info] at
> org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
> [info] at
> org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
> [info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
> [info] at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
> [info] at
> org.apache.spark.scheduler.DAGSchedulerSuite.org$scalatest$BeforeAndAfter$$super$runTest(DAGSchedulerSuite.scala:60)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
