[
https://issues.apache.org/jira/browse/FLINK-31005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-31005:
----------------------------
Description:
This task aims to verify [FLIP-281 Supports speculative execution of
sinks|https://issues.apache.org/jira/browse/FLINK-30725].
The documentation can be found
[here|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/speculative_execution/#enable-sinks-for-speculative-execution]
.
Things to verify:
1. If a sink implements the decorative interface
{{SupportsConcurrentExecutionAttempts}, Speculative executions can be
performed, otherwise not. Sinks to verify includes SinkFunction, OutputFormat
and Sink(V2).
2. These built-in sinks supports speculative execution: DiscardingSink,
PrintSinkFunction, PrintSink, FileSink, FileSystemOutputFormat, HiveTableSink
It's important to verify that the final data produced are correct and the
operator chain is as expected, i.e. no unexpected chain-break.
If it's hard to construct a case that speculative execution would happen,
especially for those built-in sinks, the speculative execution configuration
can be tuned to allow it easier to happen, e.g. set
{{slow-task-detector.execution-time.baseline-lower-bound}} and
{{slow-task-detector.execution-time.baseline-ratio}} to {{0}}.
was:
This task aims to verify [FLIP-281 Supports speculative execution of
sinks|https://issues.apache.org/jira/browse/FLINK-30725].
The documentation can be found
[here|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/speculative_execution/#enable-sinks-for-speculative-execution]
.
Things to verify:
1. If a sink implements the decorative interface
{{SupportsConcurrentExecutionAttempts}, Speculative executions can be
performed, otherwise not. Sinks to verify includes SinkFunction, OutputFormat
and Sink(V2).
2. These built-in sinks supports speculative execution: DiscardingSink,
PrintSinkFunction, PrintSink, FileSink, FileSystemOutputFormat, HiveTableSink
If it's hard to construct a case that speculative execution would happen,
especially for those built-in sinks, the speculative execution configuration
can be tuned to allow it easier to happen, e.g. set
{{slow-task-detector.execution-time.baseline-lower-bound}} and
{{slow-task-detector.execution-time.baseline-ratio}} to {{0}}.
> Release Testing: Verify FLIP-281 Supports speculative execution of sinks
> ------------------------------------------------------------------------
>
> Key: FLINK-31005
> URL: https://issues.apache.org/jira/browse/FLINK-31005
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Zhu Zhu
> Priority: Blocker
> Fix For: 1.17.0
>
>
> This task aims to verify [FLIP-281 Supports speculative execution of
> sinks|https://issues.apache.org/jira/browse/FLINK-30725].
> The documentation can be found
> [here|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/speculative_execution/#enable-sinks-for-speculative-execution]
> .
> Things to verify:
> 1. If a sink implements the decorative interface
> {{SupportsConcurrentExecutionAttempts}, Speculative executions can be
> performed, otherwise not. Sinks to verify includes SinkFunction, OutputFormat
> and Sink(V2).
> 2. These built-in sinks supports speculative execution: DiscardingSink,
> PrintSinkFunction, PrintSink, FileSink, FileSystemOutputFormat, HiveTableSink
> It's important to verify that the final data produced are correct and the
> operator chain is as expected, i.e. no unexpected chain-break.
> If it's hard to construct a case that speculative execution would happen,
> especially for those built-in sinks, the speculative execution configuration
> can be tuned to allow it easier to happen, e.g. set
> {{slow-task-detector.execution-time.baseline-lower-bound}} and
> {{slow-task-detector.execution-time.baseline-ratio}} to {{0}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
