[
https://issues.apache.org/jira/browse/FLINK-29663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lijie Wang updated FLINK-29663:
-------------------------------
Release Note:
In 1.17, we further enhanced the adaptive batch scheduler, mainly including:
(1) Support for evenly distributing data to downstream tasks
(2) Remove the limitation that the decided parallelism of vertices can only be
a power of 2
was:
In 1.17, we further enhanced the adaptive batch scheduler, mainly including:
(1) Support for evenly distributing data to downstream tasks
(2) Remove the limitation that the parallelism of vertices can only be a power
of 2
> Further improvements of adaptive batch scheduler
> ------------------------------------------------
>
> Key: FLINK-29663
> URL: https://issues.apache.org/jira/browse/FLINK-29663
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Reporter: Lijie Wang
> Assignee: Lijie Wang
> Priority: Major
> Fix For: 1.17.0
>
>
> In Flink 1.15, we introduced the adaptive batch scheduler to automatically
> decide parallelisms of job vertices for batch jobs. In this issue, we will
> further optimize it by changing the subpartition range division algorithm:
> change it from dividing according to the number of subpartitions(the number
> of subpartitions within each subpartition range is basically the same) to
> dividing according to the amount of data in subpartition ranges (the amount
> of data within each subpartition range is basically the same).
> More details see
> [https://docs.google.com/document/d/1Qyq3qDkBCUNupajVJpFTKp3fHQJtwIu9luM7T52k1Oo]
> This is the umbrella ticket for the improvements.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
