Sanket Reddy created SPARK-33741:
------------------------------------
Summary: Add minimum threshold speculation config
Key: SPARK-33741
URL: https://issues.apache.org/jira/browse/SPARK-33741
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.0.1
Reporter: Sanket Reddy
When we turn on speculation with default configs we have the last 10% of the
tasks subject to speculation. There are a lot of stages where the stage runs
for few seconds to minutes. Also in general we don't want to speculate tasks
that run within a specific interval. By setting a minimum threshold for
speculation gives us better control
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
