danny0405 commented on code in PR #13060:
URL: https://github.com/apache/hudi/pull/13060#discussion_r2021975643
##########
hudi-common/src/main/java/org/apache/hudi/common/model/PartitionBucketIndexHashingConfig.java:
##########
@@ -196,24 +196,51 @@ public static Option<PartitionBucketIndexHashingConfig>
loadHashingConfig(Hoodie
/**
* Get Latest committed hashing config instant to load.
+ * If instant is empty, then return latest hashing config instant
*/
- public static String
getLatestHashingConfigInstantToLoad(HoodieTableMetaClient metaClient) {
+ public static Option<String>
getHashingConfigInstantToLoad(HoodieTableMetaClient metaClient, Option<String>
instant) {
try {
List<String> allCommittedHashingConfig =
getCommittedHashingConfigInstants(metaClient);
- return allCommittedHashingConfig.get(allCommittedHashingConfig.size() -
1);
+ if (instant.isPresent()) {
+ Option<String> res =
getHashingConfigInstantToLoadBeforeOrOn(allCommittedHashingConfig,
instant.get());
+ // fall back to look up archived hashing config instant before return
empty
+ return res.isPresent() ? res :
getHashingConfigInstantToLoadBeforeOrOn(getArchiveHashingConfigInstants(metaClient),
instant.get());
+ } else {
+ return
Option.of(allCommittedHashingConfig.get(allCommittedHashingConfig.size() - 1));
+ }
} catch (Exception e) {
throw new HoodieException("Failed to get hashing config instant to
load.", e);
}
}
+ private static Option<String>
getHashingConfigInstantToLoadBeforeOrOn(List<String> hashingConfigInstants,
String instant) {
Review Comment:
We should always load the latest bucket config to comply with the latest
bucket id mappings. Otherwise the query would fail. We do not ensure snapshot
isolation here because the reader read being affected by the writer.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
