rohan-uptycs commented on code in PR #8503:
URL: https://github.com/apache/hudi/pull/8503#discussion_r1177555612
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/HoodieTimelineArchiver.java:
##########
@@ -509,7 +509,15 @@ private Stream<HoodieInstant> getCommitInstantsToArchive()
throws IOException {
}
private Stream<HoodieInstant> getInstantsToArchive() throws IOException {
- Stream<HoodieInstant> instants =
Stream.concat(getCleanInstantsToArchive(), getCommitInstantsToArchive());
+ List<HoodieInstant> commitInstantsToArchive =
getCommitInstantsToArchive().collect(Collectors.toList());
+ Stream<HoodieInstant> instants =
Stream.concat(getCleanInstantsToArchive(), commitInstantsToArchive.stream());
+ HoodieInstant hoodieOldestInstantToArchive =
commitInstantsToArchive.stream().max(Comparator.comparing(maxInstant ->
maxInstant.getTimestamp())).orElse(null);
+ /**
+ * if hoodieOldestInstantToArchive is null that means nothing is getting
archived, so no need to update metadata
+ */
+ if (hoodieOldestInstantToArchive != null) {
+ table.getIndex().updateMetadata(table,
Option.of(hoodieOldestInstantToArchive));
Review Comment:
@SteNicholas , understood your concern. Should we run update metadata in
postcommit of clustering operation?
https://github.com/apache/hudi/blob/b690346a700121124e2da8eb0013674ba7a0d719/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java#L522.
Will that makes sense??
As new metadata file will get created only with clustering operation in
consistent hash engine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
