kbuci commented on code in PR #18295:
URL: https://github.com/apache/hudi/pull/18295#discussion_r2964072054
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataWriteUtils.java:
##########
@@ -350,6 +388,26 @@ public static HoodieWriteConfig createMetadataWriteConfig(
}
HoodieWriteConfig metadataWriteConfig = builder.build();
+ if (deriveMetadataLockConfigsFromDataTableConfigs) {
+ // We need to update the MDT write config to have the same lock related
configs as the data table.
+ // All data table props with the lock prefix are always copied (to
override MDT defaults with
+ // user-configured values). Other data table props not present in MDT
config are also copied to
+ // support custom lock providers that may use non-standard config keys.
+ Properties lockProps = new Properties();
+ TypedProperties dataTableProps = writeConfig.getProps();
+ TypedProperties mdtProps = metadataWriteConfig.getProps();
+ for (String key : dataTableProps.stringPropertyNames()) {
Review Comment:
The issue I'm running into, which I'm discussing with @nsivabalan in
https://github.com/apache/hudi/pull/18295/#discussion_r2944943737 , is that we
need to "inherit" the lock configs from data table. But we can't directly add
all the props from data table, since that will override the non-lock related
ones that are explictly set in MDT. Originally when I first raised this PR, I
was iterating through lock provider types the same way we already handle all
metrics reporter type here. But then the issue is that this would have to be
updated anytime we add a new lock config or lock provider
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
