luoyuxia created FLINK-30556:
--------------------------------
Summary: Improve the logic for enumerating splits for Hive source
to avoid potential OOM
Key: FLINK-30556
URL: https://issues.apache.org/jira/browse/FLINK-30556
Project: Flink
Issue Type: Improvement
Components: Connectors / Hive
Affects Versions: 1.16.0
Reporter: luoyuxia
Currently, when read hive source in batch mode, it'll first enumerate all split
for the hive table. But when the table is large, the split will be too many
which it may well cause OOM. Some commuity users has also reported this
problem.
We need to optimize the logic for enumerating splits for hive table source to
avoid OOM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
