nsivabalan commented on code in PR #14008:
URL: https://github.com/apache/hudi/pull/14008#discussion_r2389104021
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java:
##########
@@ -1636,10 +1636,13 @@ public void update(HoodieRollbackMetadata
rollbackMetadata, String instantTime)
// The commit which is being rolled back on the dataset
final String commitToRollbackInstantTime =
rollbackMetadata.getCommitsRollback().get(0);
// The deltacommit that will be rolled back
- HoodieInstant deltaCommitInstant =
metadataMetaClient.createNewInstant(HoodieInstant.State.COMPLETED,
- HoodieTimeline.DELTA_COMMIT_ACTION, commitToRollbackInstantTime);
- if
(metadataMetaClient.getActiveTimeline().getDeltaCommitTimeline().containsInstant(deltaCommitInstant.requestedTime()))
{
- validateRollback(commitToRollbackInstantTime);
+ Option<HoodieInstant> deltaCommitInstantOpt =
metadataMetaClient.getActiveTimeline()
+ .getDeltaCommitTimeline()
+ .filterCompletedInstants()
+ .filter(s -> s.requestedTime().equals(commitToRollbackInstantTime))
+ .firstInstant();
+ if (deltaCommitInstantOpt.isPresent()) {
Review Comment:
let me summarize whats going on:
We are looking to apply a rollback from DT to MDT here. before streeaming
dag, we were checking for completed dc in mdt and doing a validation that the
instant of interest has to be greater than latest compaction in MDT.
w/ streaming dag fix, we ended up triggering rollback of a pending DC as
well from MDT which is the root issue we are looking to fix here.
Based on this, here is my take.
- For completed commits (completed DC's in MDT), we should have the
validation same as before.
- For those that are still pending in MDT, we should still trigger rollback,
but don't need any validations to be done.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
