linliu-code commented on code in PR #12597:
URL: https://github.com/apache/hudi/pull/12597#discussion_r1933983768
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieBaseFileGroupRecordBuffer.java:
##########
@@ -576,15 +577,25 @@ private boolean isDeleteRecordWithNaturalOrder(Option<T>
rowOption,
}
private boolean isDeleteRecord(Option<T> record, Schema schema) {
+ // Case 1: no data.
if (record.isEmpty()) {
return true;
}
-
+ // Case 2: is a delete operation.
Object operation = readerContext.getValue(record.get(), schema,
OPERATION_METADATA_FIELD);
if (operation != null &&
HoodieOperation.isDeleteRecord(operation.toString())) {
return true;
}
-
+ // Case 3: is a custom delete.
+ boolean isDeleted = HoodieRecordUtils.isCustomDeleteRecord(
Review Comment:
@nsivabalan , I have checked the code, and from what I can see: 1. During
the file read, i.e., HoodieFileReader, , since properties are not passed to
the reader (`getRecordIterator` api), currently it is impossible to add a flag
during the constructor. How to wire the properties into these readers correctly
will need some investigation. 2. During the write, there are many entry points.
how to add it will depends on how to prepare the input records. To sum, it will
be complicated for now to extract the check logic out currently. But I agree it
can bring some performance optimization, and we may need some revisit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
