danny0405 commented on code in PR #8245:
URL: https://github.com/apache/hudi/pull/8245#discussion_r1145683921
##########
hudi-sync/hudi-adb-sync/src/main/java/org/apache/hudi/sync/adb/AdbSyncTool.java:
##########
@@ -189,6 +190,13 @@ private void syncHoodieTable(String tableName, boolean
useRealtimeInputFormat, b
}
LOG.info("Last commit time synced was found:{}",
lastCommitTimeSynced.orElse("null"));
+ String currentInstantTs = config.getString(META_SYNC_CURRENT_INSTANT_TS);
+ if (lastCommitTimeSynced.isPresent() && !currentInstantTs.isEmpty()) {
+ // If occ is enabled, the commit time may be earlier than
lastCommitTimeSynced.
+ // In this case, the metadata of the current instant cannot be
synchronized, resulting in the loss of partition metadata.
+ lastCommitTimeSynced =
currentInstantTs.compareTo(lastCommitTimeSynced.get()) < 0 ?
Option.of(currentInstantTs) : lastCommitTimeSynced;
+ LOG.info("Reset lastCommitTimeSynced " + lastCommitTimeSynced.get());
Review Comment:
Somehow we need to rememer the client sync time for each writer, or keep all
the instant times that was synced in server, the timestamp comparison is tricky
and likely to produce bugs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
