squito commented on issue #24208: [SPARK-27272][CORE] Enable blacklisting of node/executor on fetch failures by default URL: https://github.com/apache/spark/pull/24208#issuecomment-476763493 I feel the same as Tom --I'm in favor of turning it on by default, but not sure about the extra config. Given that the scheduler still treats all map output as missing on the node after one fetch failure, it seems inconsistent to have this part configurable, unless we have a case supporting it. (If its going to be true by default, I think we should remove "experimental" from the docs in the `configuration.md` also). Separately, do you have any opinions on turning blacklisting on by default? maybe we should start a discussion on dev@ to get more input on that. Its been working for us for a while now, and seems critical for large clusters, so I think it would be good.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
