linliu-code commented on code in PR #9593:
URL: https://github.com/apache/hudi/pull/9593#discussion_r1340559613
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecordMerger.java:
##########
@@ -56,7 +76,8 @@ public HoodieRecordType getRecordType() {
private Option<IndexedRecord> combineAndGetUpdateValue(HoodieRecord older,
HoodieRecord newer, Schema schema, Properties props) throws IOException {
Option<IndexedRecord> previousAvroData = older.toIndexedRecord(schema,
props).map(HoodieAvroIndexedRecord::getData);
if (!previousAvroData.isPresent()) {
- return Option.empty();
+ Option<IndexedRecord> newData = newer.toIndexedRecord(schema,
props).map(HoodieAvroIndexedRecord::getData);
+ return newData;
Review Comment:
@yihua, @codope, based on the old logic, when the old record is not valid,
the merge result is empty, then the merge logic in MergeHandle will try to keep
the old data, that is, the invalid old data will be written to the disk again.
So I updated it as: when the old data is invalid, but the new one is valid, we
should output the new data; the mergeHandle will check if the new data is a
delete or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
