[
https://issues.apache.org/jira/browse/FLINK-31996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719216#comment-17719216
]
Chesnay Schepler commented on FLINK-31996:
------------------------------------------
?? So I would expect this to not be a breaking change.??
We need an opt-in flag because this change may break existing chains, which can
result in certain types being serialized that previously weren't. This can
affect performance and savepoint compatibility.
E.g., if you have a job that is one big chain then your super special custom
type doesn't go through Kryo. Users may also rely on the same object being
passed through the job.
> Chaining operators with different max parallelism prevents rescaling
> --------------------------------------------------------------------
>
> Key: FLINK-31996
> URL: https://issues.apache.org/jira/browse/FLINK-31996
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Coordination
> Reporter: David Morávek
> Priority: Major
>
> We might chain operators with different max parallelism together if they are
> set to have the same parallelism initially.
> When we decide to rescale the JobGraph vertices (using AdaptiveScheduler),
> we're gapped by the lowest maxParallelism of the operator chain. This is
> especially visible with things like CollectSink, TwoPhaseCommitSink, CDC, and
> a GlobalCommiter with maxParallelism set to 1.
>
> An obvious solution would be to prevent the chaining of operators with
> different maxParallelism, but we need to double-check this doesn't introduce
> a breaking change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
