linliu-code commented on code in PR #12597:
URL: https://github.com/apache/hudi/pull/12597#discussion_r1933983768
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieBaseFileGroupRecordBuffer.java:
##########
@@ -576,15 +577,25 @@ private boolean isDeleteRecordWithNaturalOrder(Option<T>
rowOption,
}
private boolean isDeleteRecord(Option<T> record, Schema schema) {
+ // Case 1: no data.
if (record.isEmpty()) {
return true;
}
-
+ // Case 2: is a delete operation.
Object operation = readerContext.getValue(record.get(), schema,
OPERATION_METADATA_FIELD);
if (operation != null &&
HoodieOperation.isDeleteRecord(operation.toString())) {
return true;
}
-
+ // Case 3: is a custom delete.
+ boolean isDeleted = HoodieRecordUtils.isCustomDeleteRecord(
Review Comment:
@nsivabalan , I have checked the code, and from what I can see: 1. During
the file read, i.e., HoodieFileReader, , we cannot add a flag for it since
properties are not passed to the reader (`getRecordIterator` api). How to wire
the properties into these readers will need some investigation. 2. During the
write, it will depend on how to prepare the input records. To sum, it will be
complicated for now to extract the check logic out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
