danny0405 commented on code in PR #10976:
URL: https://github.com/apache/hudi/pull/10976#discussion_r1578654464
##########
hudi-common/src/main/java/org/apache/hudi/common/table/timeline/HoodieDefaultTimeline.java:
##########
@@ -540,10 +542,10 @@ private Option<HoodieInstant>
getLastOrFirstPendingClusterInstant(boolean isLast
@Override
public boolean isPendingClusterInstant(String instantTime) {
- HoodieTimeline potentialTimeline =
getCommitsTimeline().filterPendingReplaceTimeline().filter(i ->
i.getTimestamp().equals(instantTime));
- if (potentialTimeline.countInstants() == 0) {
+ if (!getOrCreatePendingReplaceInstantSet().contains(instantTime)) {
return false;
}
+ HoodieTimeline potentialTimeline =
getCommitsTimeline().filterPendingReplaceTimeline().filter(i ->
i.getTimestamp().equals(instantTime));
if (potentialTimeline.countInstants() > 1) {
throw new IllegalStateException("Multiple instants with same timestamp:
" + potentialTimeline);
Review Comment:
Maybe we can store the cache as `instant time -> duplicate instant time
count`, so that we can always have a `O1` query for the check.
And to be clear, the cache should only cache pending clustering instant, so
`ClusteringUtils.isClusteringInstant` should be used in the filter of
`getOrCreatePendingReplaceInstantSet`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
