Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/20930#discussion_r183198368
--- Diff: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
---
@@ -1266,6 +1266,9 @@ class DAGScheduler(
}
if (failedEpoch.contains(execId) && smt.epoch <=
failedEpoch(execId)) {
logInfo(s"Ignoring possibly bogus $smt completion from
executor $execId")
+ } else if (failedStages.contains(shuffleStage)) {
--- End diff --
>Sorry I may nitpick here.
No, that's necessary, I should have to make sure about this, thanks for
your advice! :)
> Can you simulate what happens to result task if FechFaileded comes before
task success?
Sure, but it maybe hardly to reproduce this in real env, I'll try to fake
it on UT first ASAP.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
