Zhu Zhu created FLINK-28137:
-------------------------------
Summary: Introduce SpeculativeScheduler
Key: FLINK-28137
URL: https://issues.apache.org/jira/browse/FLINK-28137
Project: Flink
Issue Type: Sub-task
Components: Runtime / Coordination
Reporter: Zhu Zhu
Fix For: 1.16.0
A SpeculativeScheduler will be used if speculative execution is enabled. It
extends AdaptiveBatchScheduler so that speculative execution can work along
with the feature to adaptively tuning parallelisms for batch jobs.
The major differences of SpeculativeScheduler are:
* SpeculativeScheduler needs to be able to directly deploy an Execution, while
AdaptiveBatchScheduler can only perform ExecutionVertex level deployment.
* SpeculativeScheduler does not restart the ExecutionVertex if an execution
fails when any other current execution is still making progress
* SpeculativeScheduler listens on slow tasks. Once there are slow tasks, it
will block the slow nodes and deploy speculative executions of the slow tasks
on other nodes.
* Once any execution finishes, SpeculativeScheduler will cancel all the
remaining executions of the same execution vertex.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
