[
https://issues.apache.org/jira/browse/FLINK-35293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xingbe resolved FLINK-35293.
----------------------------
Fix Version/s: 1.20.0
Release Note:
In Flink 1.20, we have introduced support for dynamic source parallelism
inference in batch jobs for the Hive source connector. This allows the
connector to dynamically determine parallelism based on the actual partitions
with dynamic partition pruning.
Additionally, we have introduced a new configuration option,
'table.exec.hive.infer-source-parallelism.mode,' to enable users to choose
between static and dynamic inference modes for source parallelism. By default,
the mode is set to 'dynamic'. Users may configure it to 'static' for static
inference, 'dynamic' for dynamic inference, or 'none' to disable automatic
parallelism inference altogether. It should be noted that in Flink 1.20, the
previous configration option 'table.exec.hive.infer-source-parallelism' has
been marked as deprecated, but it will continue to serve as a switch for
automatic parallelism inference until it is fully phased out.
Resolution: Fixed
> FLIP-445: Support dynamic parallelism inference for HiveSource
> --------------------------------------------------------------
>
> Key: FLINK-35293
> URL: https://issues.apache.org/jira/browse/FLINK-35293
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Hive
> Affects Versions: 1.20.0
> Reporter: xingbe
> Assignee: xingbe
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.20.0
>
>
> [FLIP-379|https://cwiki.apache.org/confluence/display/FLINK/FLIP-379%3A+Dynamic+source+parallelism+inference+for+batch+jobs]
> introduces dynamic source parallelism inference, which, compared to static
> inference, utilizes runtime information to more accurately determine the
> source parallelism. The FileSource already possesses the capability for
> dynamic parallelism inference. As a follow-up task to FLIP-379, this FLIP
> plans to implement the dynamic parallelism inference interface for
> HiveSource, and also switches the default static parallelism inference to
> dynamic parallelism inference.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
