ssdong commented on a change in pull request #2784:
URL: https://github.com/apache/hudi/pull/2784#discussion_r611372415
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/utils/MetadataConversionUtils.java
##########
@@ -76,15 +76,16 @@ public static HoodieArchivedMetaEntry
createMetaWrapper(HoodieInstant hoodieInst
archivedMetaWrapper.setHoodieReplaceCommitMetadata(ReplaceArchivalHelper.convertReplaceCommitMetadata(replaceCommitMetadata));
} else if (hoodieInstant.isInflight()) {
// inflight replacecommit files have the same meta data body as
HoodieCommitMetadata
- // so we could re-use it without further creating an inflight
extension
- HoodieCommitMetadata inflightCommitMetadata = HoodieCommitMetadata
-
.fromBytes(metaClient.getActiveTimeline().getInstantDetails(hoodieInstant).get(),
HoodieCommitMetadata.class);
-
archivedMetaWrapper.setHoodieInflightReplaceMetadata(convertCommitMetadata(inflightCommitMetadata));
+ // so we could re-use it without further creating an inflight
extension.
+ // Or inflight replacecommit files are empty under clustering
circumstance
+ Option<HoodieCommitMetadata> inflightCommitMetadata =
getInflightReplaceMetadata(metaClient, hoodieInstant);
+ if (inflightCommitMetadata.isPresent()) {
+
archivedMetaWrapper.setHoodieInflightReplaceMetadata(convertCommitMetadata(inflightCommitMetadata.get()));
+ }
} else {
- // we may have some cases with empty HoodieRequestedReplaceMetadata
- // e.g. insert_overwrite_table or insert_overwrite without clustering
- Option<HoodieRequestedReplaceMetadata> requestedReplaceMetadata =
- ClusteringUtils.getRequestedReplaceMetadata(metaClient,
hoodieInstant);
+ // we may have cases with empty HoodieRequestedReplaceMetadata e.g.
insert_overwrite_table or insert_overwrite
+ // without clustering. However, we should revisit the requested
commit file standardization
+ Option<HoodieRequestedReplaceMetadata> requestedReplaceMetadata =
getRequestedReplaceMetadata(metaClient, hoodieInstant);
Review comment:
The `MetadataConversionUtils` should suffice to group such logic. They
are all about _converting_ commit files to Avro schema based metadata. 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
