cloud-fan commented on issue #26633: [SPARK-29994][CORE] Add WILDCARD task location URL: https://github.com/apache/spark/pull/26633#issuecomment-558921277 Seems we all agree that the delay scheduling is problematic. And theoretically delay scheduling can be critical for some jobs (we can easily think of a custom RDD to prove this). Setting locality wait to 0 by default is another topic. Even if we do it, people that run jobs that need delay scheduling still need to set the locality wait. For these users, we need this WILDCARD location feature to enable AQE. That said, changing the default locality wait to 0 doesn't solve the problem. It doesn't mean locality wait is always 0. As long as it can be non-zero, we need to deal with it in AQE.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
