xuanyuanking commented on issue #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#issuecomment-519114192 ``` you might have multiple attempts for the same task writing the same shuffle output, and with this approach they've even got the same shuffle generation id. ``` @squito Great thanks for your comment, also thanks for the remind by @cloud-fan, that's the scenario for a speculative task, we need to forbid running speculative task for the indeterminate stage. I'll fill a JIRA and address it separate soon.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
