Devaraj K created SPARK-13965:
---------------------------------
Summary: Driver should kill the other running task attempts if any
one task attempt succeeds for the same task
Key: SPARK-13965
URL: https://issues.apache.org/jira/browse/SPARK-13965
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.6.1
Reporter: Devaraj K
When we enable speculation, Driver would launch additional attempts for the
same task if it founds that attempt is progressing slow compared to other tasks
average progress and then there will be multiple task attempts in running state.
At present, if any one attempt gets succeeded others would be keep running
(even they could run till the job completion) and cannot be given these slots
to other tasks in same stage or in next stages.
We can kill these running task attempts when any other attempt gets succeeded
and can be given the slots to run other tasks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
