yihua commented on code in PR #19749:
URL: https://github.com/apache/hudi/pull/19749#discussion_r3867047227
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecordMerger.java:
##########
@@ -71,10 +71,11 @@ public <T> BufferedRecord<T> merge(BufferedRecord<T> older,
BufferedRecord<T> ne
if (updatedRecord == previousAvroData || updatedRecord ==
HoodieRecord.SENTINEL) {
return older;
}
- if (updatedRecord == newerAvroRecord) {
- // simply return the newer record instead of creating a new record
- return newer;
- }
+ // Do not short-circuit to `newer` when updatedRecord ==
newerAvroRecord. Some payloads
+ // (e.g. PostgresDebeziumAvroPayload's TOAST backfill via
mergeToastedValuesIfPresent)
+ // mutate the incoming Avro record in place and return the same
reference. The engine-native
+ // `newer` record is a distinct object that does not reflect those
in-place mutations, so
+ // returning it would drop the merge result. Always rebuild the result
from updatedRecord.
// Construct a new BufferedRecord with updated value
T resultRecord = recordContext.convertAvroRecord(updatedRecord);
Review Comment:
This is for correctness only for Avro-based code path used by the payload
classes in table v6. For indexing this means additional MDT record on the same
record key, which is safe. We'll revisit minor performance impact separately.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
