the-other-tim-brown commented on code in PR #13699:
URL: https://github.com/apache/hudi/pull/13699#discussion_r2280689556
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/FileGroupReaderBasedMergeHandle.java:
##########
@@ -198,8 +270,12 @@ public void doMerge() {
}
// Writes the record
try {
- writeToFile(record.getKey(), record, writeSchemaWithMetaFields,
- config.getPayloadConfig().getProps(), preserveMetadata);
+ // For Compaction operations, the preserveMetadata flag is always
true as we want to preserve the existing record metadata.
+ // For other updates, we only want to preserve the metadata if the
record is not being modified by this update. If the record already exists in
the base file and is not updated,
+ // the operation will be null. Records that are being updated or
records being added to the file group for the first time will have an operation
set and must generate new metadata.
+ boolean shouldPreserveRecordMetadata = preserveMetadata ||
record.getOperation() == null;
+ Schema recordSchema = shouldPreserveRecordMetadata ?
writeSchemaWithMetaFields : writeSchema;
+ writeToFile(record.getKey(), record, recordSchema,
config.getPayloadConfig().getProps(), shouldPreserveRecordMetadata);
Review Comment:
Create this JIRA [ticket](https://issues.apache.org/jira/browse/HUDI-9724)
to track and will take it up in the next day or so
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
