the-other-tim-brown commented on code in PR #13586:
URL: https://github.com/apache/hudi/pull/13586#discussion_r2225632172
##########
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java:
##########
@@ -427,26 +428,50 @@ private Option<Pair<HoodieInstant, HoodieCommitMetadata>>
getLatestCommitMetadat
.map(instant -> Pair.of(instant,
commitMetadataCache.get().get(instant)));
}
- private Option<Pair<HoodieInstant, HoodieCommitMetadata>>
getLatestCommitMetadataWithValidData() {
- if (latestCommitWithValidData == null) {
- Option<Pair<HoodieInstant, HoodieCommitMetadata>>
instantAndCommitMetadata =
- metaClient.getActiveTimeline().getLastCommitMetadataWithValidData();
- if (instantAndCommitMetadata.isPresent()) {
- HoodieInstant instant = instantAndCommitMetadata.get().getLeft();
- HoodieCommitMetadata metadata =
instantAndCommitMetadata.get().getRight();
+ private Option<Pair<HoodieInstant, HoodieCommitMetadata>>
getLatestCommitMetadataWithInsertOrUpdate() {
+ if (latestCommitWithValidSchema != null &&
commitMetadataCache.get().containsKey(latestCommitWithValidSchema)) {
Review Comment:
@danny0405 wanted to flag this since it is new and outside of what we were
discussing. Basically this is an optimistic check to avoid extra parsing. This
path is true in the common case where we call `hasOperationField` from
`getTableSchemaFromLatestCommitMetadata` and there is a recent commit with
inserts/updates. This should help further shave off some overhead when we
resolve the schema for the table.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
