weimingdiit commented on code in PR #8301:
URL: https://github.com/apache/hudi/pull/8301#discussion_r1160487484
##########
hudi-sync/hudi-sync-common/src/main/java/org/apache/hudi/sync/common/HoodieSyncConfig.java:
##########
@@ -163,6 +163,11 @@ public class HoodieSyncConfig extends HoodieConfig {
.defaultValue("")
.withDocumentation("The spark version used when syncing with a
metastore.");
+ public static final ConfigProperty<String> META_SYNC_PARTITION_FIXMODE =
ConfigProperty
+ .key("hoodie.datasource.hive_sync.partition_fixmode")
+ .defaultValue("false")
+ .withDocumentation("Implement a full partition sync operation when
partitions are lost.");
Review Comment:
[hoodie.datasource.hive_sync.incremental] -- This name is ok.
I think the usage scenario of this parameter is:
true :
When it is found that the partition metadata in hms is less than the actual
partition of fs, an alignment needs to be done to complete the missing
partitions in hms. Only when the metadata in hms needs to be completed, it
needs to be opened once. When the parameter is true, lastCommitTimeSynced will
be null, it is a full alignment operation at this time.
`
Option<String> lastCommitTimeSynced = Option.empty();
if (tableExists & !config.getBoolean(META_SYNC_PARTITION_FIXMODE)) {
lastCommitTimeSynced = syncClient.getLastCommitTimeSynced(tableName);
}
`
false:
Under normal circumstances, lastCommitTimeSynced is used as the baseline to
synchronize the newly generated partitions after lastCommitTimeSynced.
So the default should probably be false.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
