cloud-fan commented on issue #26975: [SPARK-30325][CORE] markPartitionCompleted cause task status inconsistent URL: https://github.com/apache/spark/pull/26975#issuecomment-570142340 If T1 finished first, but T2 is still running, and executor is lost. There are 2 cases: 1. the stage is already finished, then we will hit fetch failure later and retry stage. 2. the stage is still running. Then both TSM1 and TSM2 call `executorLost`. TSM1 will resubmit a task for P1. TSM2 may mark itself as finished. This is OK as we still have a task submitted for P1.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
