[
https://issues.apache.org/jira/browse/FLINK-31052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17691110#comment-17691110
]
Junrui Li commented on FLINK-31052:
-----------------------------------
I have tested it and it looks good to me.
I created two simple batch jobs with only source and sink operator connected by
hash edges. Job 1 has data skew, while Job 2 has no data skew but has a
taskManager that processes data very slowly. And I set the
`execution.batch.speculative.enabled` = `true` and
`slow-task-detector.execution-time.baseline-lower-bound` = `0s` and
`slow-task-detector.execution-time.baseline-ratio` = `0.1` to enable
speculative scheduler and slow task detection more qucikly.
As expected job 1 did not detect slow tasks, while job 2 did.
> Release Testing: Verify FLINK-30707 Improve slow task detection
> ---------------------------------------------------------------
>
> Key: FLINK-31052
> URL: https://issues.apache.org/jira/browse/FLINK-31052
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Zhu Zhu
> Assignee: Junrui Li
> Priority: Blocker
> Fix For: 1.17.0
>
>
> This task aims to verify
> [FLINK-30707|https://issues.apache.org/jira/browse/FLINK-30707] which
> improves the slow task detection.
> The slow task detection now takes the input data volume of tasks into
> account. Tasks which has a longer execution time but consumes more data may
> not be considered as slow. This improvement helps to eliminate the negative
> impacts of data skew on slow task detecting.
> The documentation of speculative execution can be found
> [here|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/speculative_execution/#speculative-execution]
> .
> One can verify it by creating intended data skew.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
