[
https://issues.apache.org/jira/browse/HUDI-5009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pramod Biligiri updated HUDI-5009:
----------------------------------
Description:
Currently, runMetaSync() invokes each Metastore Sync in a blocking fashion, and
iterates over the different Metastores sequentially - ([code
link|https://github.com/apache/hudi/blob/release-0.12.0/hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java#L695]
within 0.12.0 branch)
And runMetaSync() is invoked during each commit, which can lead to a slow down
of commit flow if many metastores are being synced. So enable async invocation
of runMetaSync().
was:
Currently, runMetaSync() invokes each Metastore Sync in a blocking fashion, and
iterates over the different Metastores sequentially -
([code
link|https://github.com/apache/hudi/blob/release-0.12.0/hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java#L695]
within 0.12.0 branch)
And runMetaSync() is invoked during each commit, which can lead to a slow down
of commit flow if many metastores are being synced. So enable async invocation
of runMetaSync().
> Enabling invoking runMetaSync() asynchronously
> ----------------------------------------------
>
> Key: HUDI-5009
> URL: https://issues.apache.org/jira/browse/HUDI-5009
> Project: Apache Hudi
> Issue Type: Task
> Components: meta-sync
> Reporter: Pramod Biligiri
> Priority: Minor
>
> Currently, runMetaSync() invokes each Metastore Sync in a blocking fashion,
> and iterates over the different Metastores sequentially - ([code
> link|https://github.com/apache/hudi/blob/release-0.12.0/hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java#L695]
> within 0.12.0 branch)
> And runMetaSync() is invoked during each commit, which can lead to a slow
> down of commit flow if many metastores are being synced. So enable async
> invocation of runMetaSync().
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
