[
https://issues.apache.org/jira/browse/HUDI-7069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen closed HUDI-7069.
----------------------------
Resolution: Fixed
Fixed via master branch: dcd5a8182a11faab8bfc1ce8aa7787fa590dd395
> Optimize metaclient construction and include table config in write config for
> multi-table services.
> ---------------------------------------------------------------------------------------------------
>
> Key: HUDI-7069
> URL: https://issues.apache.org/jira/browse/HUDI-7069
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ma Jian
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.0.0
>
>
> In the current implementation of run multi tables services, the clustering
> task and compaction task both build metaclient repeatedly for each table,
> causing additional overhead. To reduce this overhead, we extract the
> construction of metaclient and only construct it once for each table, passing
> it as a parameter to the corresponding task.
> At the same time, when running multi tables services, the write config lacks
> some information from the table config, such as the table name. This leads to
> empty strings when retrieving the table name in certain situations. For
> example, when configuring the prefix for metrics, if not specified, the table
> name is used as the prefix. However, in the current situation, without the
> table config, it's impossible to differentiate the metrics of different
> tables, resulting in an empty prefix. By adding the table config to the write
> config beforehand, we can obtain all the configuration information in the
> subsequent write config step.
> Additionally, we made a small modification by removing the redundant
> construction of metaclient in the clusteringJob's constructor.
> !https://intranetproxy.alipay.com/skylark/lark/0/2023/png/62256341/1699595853317-03416d85-8a25-4fbd-96af-f351f0ac6ec7.png?x-oss-process=image%2Fresize%2Cw_1322%2Climit_0|width=470,height=355!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
