alexeykudinkin commented on code in PR #7759:
URL: https://github.com/apache/hudi/pull/7759#discussion_r1089555248
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecordMerger.java:
##########
@@ -71,10 +71,10 @@ public HoodieRecordType getRecordType() {
return HoodieRecordType.AVRO;
}
- private HoodieRecord preCombine(HoodieRecord older, HoodieRecord newer) {
- HoodieRecordPayload picked = unsafeCast(((HoodieAvroRecord)
newer).getData().preCombine(((HoodieAvroRecord) older).getData()));
- if (picked instanceof HoodieMetadataPayload) {
- // NOTE: HoodieMetadataPayload return a new payload
+ private HoodieRecord preCombine(HoodieRecord older, HoodieRecord newer,
Schema schema, Properties props) {
+ HoodieRecordPayload picked = unsafeCast(((HoodieAvroRecord)
newer).getData().preCombine(((HoodieAvroRecord) older).getData(), schema,
props));
+ if (picked instanceof HoodieMetadataPayload || picked instanceof
OverwriteNonDefaultsWithLatestAvroPayload) {
+ // NOTE: When payload class returns a new payload instead of choosing
from older/newer payload, create a new record.
return new HoodieAvroRecord(newer.getKey(), picked,
newer.getOperation());
}
return picked.equals(((HoodieAvroRecord) newer).getData()) ? newer : older;
Review Comment:
I think we can nuke this optimization avoiding creation of the new record
and just always create it in that case (to make sure this doesn't affect all
random payloads out there)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
