[
https://issues.apache.org/jira/browse/HUDI-1292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17398110#comment-17398110
]
ASF GitHub Bot commented on HUDI-1292:
--------------------------------------
leesf commented on a change in pull request #3427:
URL: https://github.com/apache/hudi/pull/3427#discussion_r687492662
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java
##########
@@ -44,6 +44,13 @@
.sinceVersion("0.7.0")
.withDocumentation("Enable the internal metadata table which serves
table metadata like level file listings");
+ // Enable syncing the Metadata Table
+ public static final ConfigProperty<Boolean> METADATA_SYNC_ENABLE_PROP =
ConfigProperty
Review comment:
I think enable multi-writer mode for metadata table is a better
solution. And I go through the codebase of metadata sync, in `preWrite`,
syncing metadata is in transaction(and should be ok in multi writing pipeline?)
and I see the difference between setting the `hoodie.metadata.enable` to
`false` and `hoodie.metadata.sync.enable` to `false` is creating
`SparkHoodieBackedTableMetadataWriter`, while in `postWrite`, it is not in
transaction(should not ok in multi writing pipeline). Would you please correct
me in some other aspect I missed? @prashantwason
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
> [Umbrella] RFC-15 : File Listing and Query Planning Optimizations
> ------------------------------------------------------------------
>
> Key: HUDI-1292
> URL: https://issues.apache.org/jira/browse/HUDI-1292
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Spark Integration, Writer Core
> Affects Versions: 0.9.0
> Reporter: Vinoth Chandar
> Assignee: Prashant Wason
> Priority: Major
> Labels: hudi-umbrellas, pull-request-available
> Fix For: 0.10.0
>
>
> This is the umbrella ticket that tracks the overall implementation of RFC-15
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
