cloud-fan commented on issue #24375: [SPARK-27474][CORE] try best to not submit tasks when the partitions are already completed URL: https://github.com/apache/spark/pull/24375#issuecomment-483637285 > But, doesn't #21131 could do it either ? What's the advantage compares to #21131 ? #21131 did not cover the corner case, that's why we open #23871 . Do I miss something? Maybe you were asking why I prefer this PR over #23871 . The reason is, we don't need to sync the states between `DAGScheduler` and `TaskSchedulerImpl`, which is fragile to me.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
