seayoun commented on issue #26975: [SPARK-30325][CORE] markPartitionCompleted cause task status inconsistent URL: https://github.com/apache/spark/pull/26975#issuecomment-569574798 @jiangxb1987 @Ngone51 For case 1, I think it is acceptable, if T1 and T2 both finished, the E1 and E2 have the same chance to get lost, if E2 get lost, we can `Resubmit` at current stage; however if E1 got lost, we can perceive only by next stage got `FetchFailedException` to rescheduler the partition, it is high cost. For case 2, I has the save question as @Ngone51
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
