ankuriitg commented on issue #24208: [SPARK-27272][CORE] Enable blacklisting of node/executor on fetch failures by default URL: https://github.com/apache/spark/pull/24208#issuecomment-476788386 Thanks for the review @squito ! I have removed the additional logic and configuration and changed the PR to just enable blacklisting for fetch failure to on by default. As @tgravescs suggested, we can add that logic later on, if needed. I also wanted to check if we should enable `spark.files.fetchFailure.unRegisterOutputOnHost` by default (added in SPARK-19753). This ensures that not only is the host blacklisted but all the files on that host are deleted. @squito , I also agree with turning blacklisting on by default.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
