liupc commented on issue #23647: [SPARK-26712]Support multi directories for executor shuffle info recovery in yarn shuffle serivce URL: https://github.com/apache/spark/pull/23647#issuecomment-458802178 @squito > that still doesn't completely handle the problem, as any existing shuffle data written to the bad disks is gone Yes, the existing shuffle data written for those finished `ShuffleMapTask` would gone, but Spark'retry( I mean the stage retry) mechanism should fix things, is it?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
