xingbe created FLINK-34143:
------------------------------
Summary: Modify the effective strategy of
`execution.batch.adaptive.auto-parallelism.default-source-parallelism`
Key: FLINK-34143
URL: https://issues.apache.org/jira/browse/FLINK-34143
Project: Flink
Issue Type: Improvement
Components: Runtime / Coordination
Affects Versions: 1.19.0
Reporter: xingbe
Currently, if users do not set the
`{{{}execution.batch.adaptive.auto-parallelism.default-source-parallelism`{}}}
configuration option, the AdaptiveBatchScheduler defaults to a parallelism of 1
for source vertices. In
[FLIP-379|https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs#FLIP379:Dynamicsourceparallelisminferenceforbatchjobs-IntroduceDynamicParallelismInferenceinterfaceforSource],
the value of
`{{{}execution.batch.adaptive.auto-parallelism.default-source-parallelism`{}}}
will act as the upper bound for inferring dynamic source parallelism, and
continuing with the current policy is no longer appropriate.
We plan to change the effectiveness strategy of
`{{{}execution.batch.adaptive.auto-parallelism.default-source-parallelism`{}}};
when the user does not set this config option, we will use the value of
`{{{}execution.batch.adaptive.auto-parallelism.max-parallelism`{}}} as the
upper bound for source parallelism inference. If
{{`execution.batch.adaptive.auto-parallelism.max-parallelism`}} is also not
configured, the value of `{{{}parallelism.default`{}}} will be used as a
fallback.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
