Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22004#discussion_r207753682 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2369,39 +2369,12 @@ class DAGSchedulerSuite extends SparkFunSuite with LocalSparkContext with TimeLi assert(scheduler.getShuffleDependencies(rddE) === Set(shuffleDepA, shuffleDepC)) } - test("SPARK-17644: After one stage is aborted for too many failed attempts, subsequent stages" + + test("SPARK-17644: After one stage is aborted for too many failed attempts, subsequent stages " + "still behave correctly on fetch failures") { - // Runs a job that always encounters a fetch failure, so should eventually be aborted --- End diff -- If you move ``` object FailThisAttempt { val _fail = new AtomicBoolean(true) } ``` outside tests as a top object, tests pass, no need to move the functions to the companion object. Btw the closure cleaner does not look into the body of the lambda to check if references of other objects create an issue. This is done only for the old closures. According to document we only checked for the return statements. Also Lambdas dont have outers by definition. Regarding the LegacyAccumulatorWrapper there is no closure
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org