redsanket opened a new pull request #30710: URL: https://github.com/apache/spark/pull/30710
### What changes were proposed in this pull request? Add min threshold time speculation config ### Why are the changes needed? When we turn on speculation with default configs we have the last 10% of the tasks subject to speculation. There are a lot of stages where the stage runs for few seconds to minutes. Also in general we don't want to speculate tasks that run within a minimum threshold. By setting a minimum threshold for speculation config gives us better control for speculative tasks ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Manual, can add unit test if this is not trivial change ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
