xingbe created FLINK-32288:
------------------------------
Summary: Improve the scheduling performance of
AdaptiveBatchScheduler
Key: FLINK-32288
URL: https://issues.apache.org/jira/browse/FLINK-32288
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.18.0
Reporter: xingbe
Fix For: 1.18.0
After adding the benchmark of AdaptiveBatchScheduler in FLINK-30480, we noticed
a regression in the performance of
SchedulingDownstreamTasksInBatchJobBenchmark#SchedulingDownstreamTasks. When
scheduling a batch job with a parallelism of 4000*4000, the time spent
increased from 32ms to 1336ms on my local PC.
To improve the performance, we can optimize the traversal by checking if the
consumedPartitionGroups have finished all its partitions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
