Lijie Wang created FLINK-29663:
----------------------------------
Summary: Further improvements of adaptive batch scheduler
Key: FLINK-29663
URL: https://issues.apache.org/jira/browse/FLINK-29663
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Reporter: Lijie Wang
Assignee: Lijie Wang
Fix For: 1.17.0
In Flink 1.15, we introduced the adaptive batch scheduler to automatically
decide parallelisms of job vertices for batch jobs. In this issue, we will
further optimize it by changing the subpartition range division algorithm:
change it from dividing according to the number of subpartitions(the number of
subpartitions within each subpartition range is basically the same) to dividing
according to the amount of data in subpartition ranges (the amount of data
within each subpartition range is basically the same).
More details see
[https://docs.google.com/document/d/1Qyq3qDkBCUNupajVJpFTKp3fHQJtwIu9luM7T52k1Oo]
This is the umbrella ticket for the improvements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
