cloud-fan commented on code in PR #50630:
URL: https://github.com/apache/spark/pull/50630#discussion_r2087349665
##########
core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala:
##########
@@ -1552,6 +1552,20 @@ private[spark] class DAGScheduler(
// `findMissingPartitions()` returns all partitions every time.
stage match {
case sms: ShuffleMapStage if stage.isIndeterminate && !sms.isAvailable =>
+ // already executed atleast once
Review Comment:
Can we say more in the comments? So this case is for when the shuffle stage
was not failed due to FetchFailed, we also need to roll back the result stage?
BTW will it cause repeated result stage rollback? e.g. we roll back it once
when hitting FetchFailed, and roll back it again here then the failed shuffle
stage is resubmitted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
