yihua commented on a change in pull request #2400:
URL: https://github.com/apache/hudi/pull/2400#discussion_r762399404
##########
File path:
hudi-utilities/src/main/java/org/apache/hudi/utilities/deltastreamer/DeltaSync.java
##########
@@ -307,7 +307,10 @@ public void refreshTimeline() throws IOException {
if (!commitMetadata.getMetadata(CHECKPOINT_KEY).isEmpty()) {
resumeCheckpointStr =
Option.of(commitMetadata.getMetadata(CHECKPOINT_KEY));
}
- } else if
(HoodieTimeline.compareTimestamps(HoodieTimeline.FULL_BOOTSTRAP_INSTANT_TS,
+ } else if (commitMetadata.getOperationType() ==
WriteOperationType.CLUSTER) {
+ // incase of CLUSTER commit, no checkpoint will be available in
metadata.
Review comment:
`This should be fine since the commit timeline to read the CHECKPOINT
from does not include the clustering instants.`
@n3nash @nsivabalan This is not correct. The commit timeline does have the
clustering instants (`replacecommit`) based on the changes in #2048. So this
breaks the logic of getting the last checkpoint, even if we have the walk-back
logic in #4034, which is skipped.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
