yihua commented on code in PR #12772:
URL: https://github.com/apache/hudi/pull/12772#discussion_r2074104278
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/DefaultSparkRecordMerger.java:
##########
@@ -73,7 +75,7 @@ public Option<Pair<HoodieRecord, Schema>> merge(HoodieRecord
older, Schema oldSc
return Option.of(Pair.of(newer, newSchema));
}
}
- if (older.getOrderingValue(oldSchema,
props).compareTo(newer.getOrderingValue(newSchema, props)) > 0) {
+ if (compareValues(older.getOrderingValue(oldSchema, props),
newer.getOrderingValue(newSchema, props)) > 0) {
Review Comment:
`compareValues` incurs `SparkAdapterSupport$.MODULE$.sparkAdapter()` which
incurs reflection and new instance of the adapter class per record which has
too much overhead. Let's revert it. If this is for new variant type, let's
thrown an error for such a type in this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
