Yuchen Huo created SPARK-29976:
----------------------------------
Summary: Allow speculation even if there is only one task
Key: SPARK-29976
URL: https://issues.apache.org/jira/browse/SPARK-29976
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 3.0.0
Reporter: Yuchen Huo
In the current speculative execution implementation if there is only one task
in the stage then no speculative run would be conducted. However, there might
be cases where an executor have some problem in writing to its disk and just
hang forever. In this case, if the single task stage get assigned to the
problematic executor then the whole job would hang forever. It would be better
if we could run the task on another executor if this happens.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
