liupc commented on issue #23614: [SPARK-26689][CORE]Support blacklisting bad disk directory and retry in DiskBlockManager URL: https://github.com/apache/spark/pull/23614#issuecomment-458386221 @squito Yes, it's turned on. I think the exception I attached is not the business of application level blacklist(it only handles task failure), this is a driver side exception, even before `TaskSetManager` created. So this PR just introduced disk directories blacklist to handle this case which can avoid broadcast failure and shuffle retry when encountered disk error. This benifits the applications and make them more stable, and everything works as usual even before SREs fix this disk.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
