yuchenhuo commented on issue #26614: [SPARK-29976][CORE] New conf for single task stage speculation URL: https://github.com/apache/spark/pull/26614#issuecomment-558362873 @tgravescs @squito Looks like there are two potential solutions (1) we check if all of the tasks are running on the same executor first and if so do the time threshold speculative run. (2) add another conf specifying the minimum number of total tasks in the stage that we would trigger the time threshold check. I think both of the solutions solve the problem I'm hitting but option (2) seems more configurable and handles slightly more corner cases (i.e. multiple problematic nodes).
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
