danny0405 commented on code in PR #19769:
URL: https://github.com/apache/hudi/pull/19769#discussion_r3871495666
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/BufferedRecordMergerFactory.java:
##########
@@ -512,6 +512,16 @@ private static <T> boolean
shouldKeepNewerRecord(BufferedRecord<T> oldRecord, Bu
// The orderingVal is constant 0 (int) and not guaranteed to match the
type of the old or new record's ordering value.
return true;
}
- return
newRecord.getOrderingValue().compareTo(oldRecord.getOrderingValue()) >= 0;
+ Comparable oldOrderingVal = oldRecord.getOrderingValue();
+ Comparable newOrderingVal = newRecord.getOrderingValue();
+ // Checks the ordering value does not equal to 0
+ // because we use 0 as the default value which means natural order.
+ // OrderingValues#create coerces a null ordering field value to that
default, so its class need
Review Comment:
should we even allows null ordering values for non-deletes records, for
records from `DELETE` statement, it is clear since it is indeed natural order
or commit time order, but for regular records, we can not decide the ordering
with a null ordering value which is ambiguous.
I see some code paths keep the null values(Flink) while some code paths
reset nulls as 0(spark), should we just abandon the reset?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
