[
https://issues.apache.org/jira/browse/FLINK-23826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420218#comment-17420218
]
Zhu Zhu edited comment on FLINK-23826 at 9/26/21, 6:48 AM:
-----------------------------------------------------------
Here is the benchmark results of release-1.14 v.s. release-1.12.
||version||JM/Scheduler Initializing||Task deployment||Making failure recovery
decision||
|release-1.14|0.2s|16s|0.2s|
|release-1.12|71s|128s|37s|
The benchmark uses an unbounded WordCount-like job, i.e. Source --keyBy--> Sink.
The parallelism is 10000 for all job vertices.
We compare release-1.14 to release-1.12 because some of the FLINK-21110
improvements were merged in 1.13.
was (Author: zhuzh):
Here is the benchmark results of release-1.14 v.s. release-1.12.
||version||JM/Scheduler Initializing||Task deployment||Making failure recovery
decision||
|release-1.14|0.2s|16s|0.2s|
|release-1.12|71s|128s|37s|
We compare release-1.14 to release-1.12 because some of the FLINK-21110
improvements were merged in 1.13.
> Verify optimized scheduler performance for large-scale jobs
> -----------------------------------------------------------
>
> Key: FLINK-23826
> URL: https://issues.apache.org/jira/browse/FLINK-23826
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.14.0
> Reporter: Zhu Zhu
> Assignee: Zhu Zhu
> Priority: Major
> Fix For: 1.14.0
>
>
> This ticket is used to verify the result of FLINK-21110.
> It should check if large scale jobs' scheduling are working well and the
> scheduling performance, with a real job running on cluster.
> The conclusion should include, for a *10000 --- all-to-all-connected
> -->10000* job:
> 1. time of job initialization on master (job received -> scheduling started)
> 2. time of task deployment (task deploying started -> all tasks in RUNNING)
> 3. time of making task failure recovery decision (JM notified about task
> failure -> tasks to restart decided)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
