watermelon12138 commented on code in PR #5199:
URL: https://github.com/apache/hudi/pull/5199#discussion_r853671852
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java:
##########
@@ -372,6 +372,8 @@ public void completeCompaction(
} finally {
this.txnManager.endTransaction(Option.of(compactionInstant));
}
+ WriteMarkersFactory.get(config.getMarkersType(), table,
compactionCommitTime)
+ .quietDeleteMarkerDir(context, config.getMarkersDeleteParallelism());
Review Comment:
Thank you for your review. Generally, Delete the remaining temporary marker
files in postCommit stage. So, I think compaction progress also need to delete
marker files after completing inflight compaction.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java:
##########
@@ -165,7 +165,7 @@ public Schema getTableAvroSchema(boolean
includeMetadataFields) throws Exception
if (includeMetadataFields) {
return HoodieAvroUtils.addMetadataFields(schemaFromTableConfig.get(),
hasOperationField);
} else {
- return schemaFromTableConfig.get();
+ return
HoodieAvroUtils.removeMetadataFields(schemaFromTableConfig.get());
Review Comment:
Hello, When create schema already contains metadata fields but
includeMetadataField is set to false, can avro schema that does not contain
metadata fields be returned? Considering the above scenario, I modified the
code on line 168.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/TableSchemaResolver.java:
##########
@@ -165,7 +165,7 @@ public Schema getTableAvroSchema(boolean
includeMetadataFields) throws Exception
if (includeMetadataFields) {
return HoodieAvroUtils.addMetadataFields(schemaFromTableConfig.get(),
hasOperationField);
} else {
- return schemaFromTableConfig.get();
+ return
HoodieAvroUtils.removeMetadataFields(schemaFromTableConfig.get());
Review Comment:
I answered your questions in the comments and hope you can review my code
again.
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/HoodieFlinkWriteClient.java:
##########
@@ -372,6 +372,8 @@ public void completeCompaction(
} finally {
this.txnManager.endTransaction(Option.of(compactionInstant));
}
+ WriteMarkersFactory.get(config.getMarkersType(), table,
compactionCommitTime)
+ .quietDeleteMarkerDir(context, config.getMarkersDeleteParallelism());
Review Comment:
I answered your questions in the comments and hope you can review my code
again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
