David Morávek created FLINK-31996:
-------------------------------------
Summary: Chaining operators with different max parallelism
prevents rescaling
Key: FLINK-31996
URL: https://issues.apache.org/jira/browse/FLINK-31996
Project: Flink
Issue Type: Bug
Components: Runtime / Coordination
Reporter: David Morávek
We might chain operators with different max parallelism together if they are
set to have the same parallelism initially.
When we decide to rescale the JobGraph vertices (using AdaptiveScheduler),
we're gapped by the lowest maxParallelism of the operator chain. This is
especially visible with things like CollectSink, TwoPhaseCommitSink, CDC, and a
GlobalCommiter with maxParallelism set to 1.
An obvious solution would be to prevent the chaining of operators with
different maxParallelism, but we need to double-check this doesn't introduce a
breaking change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
