[
https://issues.apache.org/jira/browse/FLINK-31996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler closed FLINK-31996.
------------------------------------
Fix Version/s: 1.18.0
Resolution: Fixed
master: 58a849bcc497e17bf575e49946f106030f3e1a1a
> Chaining operators with different max parallelism prevents rescaling
> --------------------------------------------------------------------
>
> Key: FLINK-31996
> URL: https://issues.apache.org/jira/browse/FLINK-31996
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Reporter: David Morávek
> Assignee: David Morávek
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.18.0
>
>
> We might chain operators with different max parallelism together if they are
> set to have the same parallelism initially.
> When we decide to rescale the JobGraph vertices (using AdaptiveScheduler),
> we're gapped by the lowest maxParallelism of the operator chain. This is
> especially visible with things like CollectSink, TwoPhaseCommitSink, CDC, and
> a GlobalCommiter with maxParallelism set to 1.
>
> An obvious solution would be to prevent the chaining of operators with
> different maxParallelism, but we need to double-check this doesn't introduce
> a breaking change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
