yihua commented on code in PR #12390:
URL: https://github.com/apache/hudi/pull/12390#discussion_r1866876462
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieBaseFileGroupRecordBuffer.java:
##########
@@ -495,14 +502,24 @@ protected boolean hasNextBaseRecord(T baseRecord,
Pair<Option<T>, Map<String, Ob
Map<String, Object> metadata = readerContext.generateMetadataForRecord(
baseRecord, readerSchema);
- Option<T> resultRecord = logRecordInfo != null
- ? merge(Option.of(baseRecord), metadata, logRecordInfo.getLeft(),
logRecordInfo.getRight())
- : merge(Option.empty(), Collections.emptyMap(), Option.of(baseRecord),
metadata);
- if (resultRecord.isPresent()) {
- nextRecord = readerContext.seal(resultRecord.get());
- return true;
+ if (logRecordInfo != null) {
+ Option<T> resultRecord = merge(Option.of(baseRecord), metadata,
logRecordInfo.getLeft(), logRecordInfo.getRight());
+ if (resultRecord.isPresent()) {
+ // Updates
+ nextRecord = readerContext.seal(resultRecord.get());
+ readerStats.incrementNumUpdates();
+ return true;
+ } else {
+ // Deletes
+ readerStats.incrementNumDeletes();
+ return false;
+ }
}
- return false;
+
+ // Inserts
+ nextRecord = baseRecord;
+ readerStats.incrementNumInserts();
Review Comment:
These is the case where the record in the base file is not updated in the
log files (i.e., the record key of the base file record does not exist in the
log files).
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieBaseFileGroupRecordBuffer.java:
##########
@@ -495,14 +502,24 @@ protected boolean hasNextBaseRecord(T baseRecord,
Pair<Option<T>, Map<String, Ob
Map<String, Object> metadata = readerContext.generateMetadataForRecord(
baseRecord, readerSchema);
- Option<T> resultRecord = logRecordInfo != null
- ? merge(Option.of(baseRecord), metadata, logRecordInfo.getLeft(),
logRecordInfo.getRight())
- : merge(Option.empty(), Collections.emptyMap(), Option.of(baseRecord),
metadata);
- if (resultRecord.isPresent()) {
- nextRecord = readerContext.seal(resultRecord.get());
- return true;
+ if (logRecordInfo != null) {
+ Option<T> resultRecord = merge(Option.of(baseRecord), metadata,
logRecordInfo.getLeft(), logRecordInfo.getRight());
+ if (resultRecord.isPresent()) {
+ // Updates
+ nextRecord = readerContext.seal(resultRecord.get());
+ readerStats.incrementNumUpdates();
+ return true;
+ } else {
+ // Deletes
+ readerStats.incrementNumDeletes();
+ return false;
+ }
}
- return false;
+
+ // Inserts
+ nextRecord = baseRecord;
+ readerStats.incrementNumInserts();
Review Comment:
This is the case where the record in the base file is not updated in the log
files (i.e., the record key of the base file record does not exist in the log
files).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
