[
https://issues.apache.org/jira/browse/FLINK-28980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-28980:
----------------------------
Description:
Speculative execution is introduced in Flink 1.16 to deal with temporary slow
tasks caused by slow nodes. More details about this feature can be found in
this documentation [PR|https://github.com/apache/flink/pull/20507].
This feature currently consists of 4 FLIPs:
- FLIP-168: Speculative Execution core part
- FLIP-224: Blocklist Mechanism
- FLIP-245: Source Supports Speculative Execution
- FLIP-249: Flink Web UI Enhancement for Speculative Execution
This ticket aims to verify FLIP-168, along with FLIP-224 and FLIP-249.
To do the verification, the process can be:
- Write a Flink job which has a subtask running much slower than others (e.g.
sleep indefinitely if it runs on a certain host, the hostname can be retrieved
via InetAddress.getLocalHost().getHostName(), or if its (subtaskIndex +
attemptNumer) % 2 == 0)
- Modify Flink configuration file to enable speculative execution and tune the
configuration as you like
- Submit the job. Checking the web UI, logs, metrics and produced result.
was:
Speculative execution is introduced in Flink 1.16 to deal with temporary slow
tasks caused by slow nodes. More details about this feature can be found in
this documentation [PR|https://github.com/apache/flink/pull/20507].
This feature currently consists of 4 FLIPs:
- FLIP-168: Speculative Execution core part
- FLIP-224: Blocklist Mechanism
- FLIP-245: Source Supports Speculative Execution
- FLIP-249: Flink Web UI Enhancement for Speculative Execution
This ticket aims to verify FLIP-168, along with FLIP-224 and FLIP-249.
To do the verification, the process can be:
- Write a Flink job which has a subtask running much slower than others (e.g.
sleep indefinitely if it runs on a certain host, the hostname can be retrieved
via InetAddress.getLocalHost().getHostName(), or if its (subtaskIndex +
attemptNumer) % 2 == 0)
- Modify Flink configuration file to enable speculative execution and tune the
configuration as you like
- Submit the job. Checking the web UI, logs, metrics and produced result.
> Release Testing: Verify FLIP-168 speculative execution
> ------------------------------------------------------
>
> Key: FLINK-28980
> URL: https://issues.apache.org/jira/browse/FLINK-28980
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Zhu Zhu
> Priority: Major
> Fix For: 1.16.0
>
>
> Speculative execution is introduced in Flink 1.16 to deal with temporary slow
> tasks caused by slow nodes. More details about this feature can be found in
> this documentation [PR|https://github.com/apache/flink/pull/20507].
> This feature currently consists of 4 FLIPs:
> - FLIP-168: Speculative Execution core part
> - FLIP-224: Blocklist Mechanism
> - FLIP-245: Source Supports Speculative Execution
> - FLIP-249: Flink Web UI Enhancement for Speculative Execution
> This ticket aims to verify FLIP-168, along with FLIP-224 and FLIP-249.
> To do the verification, the process can be:
> - Write a Flink job which has a subtask running much slower than others
> (e.g. sleep indefinitely if it runs on a certain host, the hostname can be
> retrieved via InetAddress.getLocalHost().getHostName(), or if its
> (subtaskIndex + attemptNumer) % 2 == 0)
> - Modify Flink configuration file to enable speculative execution and tune
> the configuration as you like
> - Submit the job. Checking the web UI, logs, metrics and produced result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
