[
https://issues.apache.org/jira/browse/HUDI-5066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan closed HUDI-5066.
-------------------------------------
Resolution: Fixed
> Support hoodie source metaclient cache for flink planner
> --------------------------------------------------------
>
> Key: HUDI-5066
> URL: https://issues.apache.org/jira/browse/HUDI-5066
> Project: Apache Hudi
> Issue Type: Improvement
> Components: flink-sql, performance
> Reporter: Shizhi Chen
> Assignee: Shizhi Chen
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.12.2, 0.13.0
>
>
> h2. Change Logs
> Flink Table Planner will invoke `HoodieTableSource.copy()` when applying its
> `RelOptRule` such as
> `PushPartitionIntoTableSourceScanRule`, thus result in multiple meta client
> instantiation, which will
> affect the performance of starting job quickly.
> {code:java}
> // apply push down
> DynamicTableSource dynamicTableSource =
> tableSourceTable.tableSource().copy();
> PartitionPushDownSpec partitionPushDownSpec =
> new PartitionPushDownSpec(remainingPartitions);
> partitionPushDownSpec.apply(dynamicTableSource,
> SourceAbilityContext.from(scan));
> {code}
> Here we promote that meta client might be cached to reused.
> h2. Impact
> speed up flink sql job start by avoiding unnecessary meta client repeated
> creation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
