nsivabalan commented on a change in pull request #4530:
URL: https://github.com/apache/hudi/pull/4530#discussion_r781161131
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metadata/HoodieBackedTableMetadataWriter.java
##########
@@ -689,7 +689,7 @@ protected void compactIfNecessary(AbstractHoodieWriteClient
writeClient, String
String latestDeltacommitTime =
metadataMetaClient.reloadActiveTimeline().getDeltaCommitTimeline().filterCompletedInstants().lastInstant()
.get().getTimestamp();
List<HoodieInstant> pendingInstants =
dataMetaClient.reloadActiveTimeline().filterInflightsAndRequested()
-
.findInstantsBefore(latestDeltacommitTime).getInstants().collect(Collectors.toList());
+
.findInstantsBefore(instantTime).getInstants().collect(Collectors.toList());
Review comment:
let me try to explain.
lets say we have 10 commits, C1, C2 -> C10.
Prior to this patch, we will compact immediately after C10 and so compaction
commit will be C10 + "001".
With this patch, we will be compacting just before C11 starts getting
applied to MDT.
And so, I am basing the compaction commit of latest delta commit time which
is C10 and not instant time which is C11.
And so, its C10 + "001". but if I go with instantTime, then we might change
the behavior. In fact, we can't do that, since compaction time will be less
than delta commit which will be eventually created when we apply C11 to MDT.
Let me know if this makes sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
