jonvex commented on code in PR #12122:
URL: https://github.com/apache/hudi/pull/12122#discussion_r1809645021
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieBaseFileGroupRecordBuffer.java:
##########
@@ -278,14 +283,16 @@ protected Option<DeleteRecord>
doProcessNextDeletedRecord(DeleteRecord deleteRec
if
(isDeleteRecordWithNaturalOrder(existingRecordMetadataPair.getLeft(),
existingOrderingVal)) {
return Option.empty();
}
- Comparable deleteOrderingVal = deleteRecord.getOrderingValue();
- // Checks the ordering value does not equal to 0
- // because we use 0 as the default value which means natural order
- boolean chooseExisting = !deleteOrderingVal.equals(0)
- && ReflectionUtils.isSameClass(existingOrderingVal,
deleteOrderingVal)
- && existingOrderingVal.compareTo(deleteOrderingVal) > 0;
- if (chooseExisting) {
- // The DELETE message is obsolete if the old message has greater
orderingVal.
+ Comparable deleteOrderingVal = deleteRecord.getOrderingValue() ==
null ? orderingFieldDefault : deleteRecord.getOrderingValue();
+ // Here existing record represents newer record with the same key,
which can be a delete or non-delete record.
+ // Therefore, we should use event time based merging if possible.
So, the newer record is returned if
+ // 1. the delete is processing time based, or
+ // 2. delete is event time based, and the existing record has higher
value.
+ if (isProcessingTimeBasedDelete(deleteOrderingVal)) {
+
existingRecordMetadataPair.getRight().put(DELETE_FOUND_WITHOUT_ORDERING_VALUE,
"true");
+ return Option.empty();
+ }
+ if (ReflectionUtils.isSameClass(existingOrderingVal,
deleteOrderingVal) && deleteOrderingVal.compareTo(existingOrderingVal) <= 0) {
Review Comment:
In get ordering value we do the casting so they should all be the same type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
