cshuo commented on code in PR #13366:
URL: https://github.com/apache/hudi/pull/13366#discussion_r2113010635
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/HoodieFileGroupReader.java:
##########
@@ -181,17 +176,15 @@ private FileGroupRecordBuffer<T>
getRecordBuffer(HoodieReaderContext<T> readerCo
RecordMergeMode
recordMergeMode,
TypedProperties props,
Option<HoodieBaseFile>
baseFileOption,
- boolean hasNoLogFiles,
+ boolean hasLogFiles,
boolean isSkipMerge,
boolean
shouldUseRecordPosition,
HoodieReadStats readStats,
boolean emitDelete) {
- if (hasNoLogFiles) {
- return null;
- } else if (isSkipMerge) {
+ if (isSkipMerge) {
Review Comment:
For this case, `UnmergedFileGroupRecordBuffer` is reused to read base file
only in `doHasNext`.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/FileGroupRecordBuffer.java:
##########
@@ -552,31 +607,34 @@ private Schema getSchemaForAvroPayloadMerge(HoodieRecord
record, BufferedRecord<
return readerContext.getSchemaFromBufferRecord(bufferedRecord);
}
- protected boolean hasNextBaseRecord(T baseRecord, BufferedRecord<T>
logRecordInfo) throws IOException {
- if (logRecordInfo != null) {
- BufferedRecord<T> baseRecordInfo =
BufferedRecord.forRecordWithContext(baseRecord, readerSchema, readerContext,
orderingFieldName, false);
- Pair<Boolean, T> isDeleteAndRecord = merge(baseRecordInfo,
logRecordInfo);
- if (!isDeleteAndRecord.getLeft()) {
- // Updates
- nextRecord = readerContext.seal(isDeleteAndRecord.getRight());
- readStats.incrementNumUpdates();
- return true;
- } else if (emitDelete) {
- // emit Deletes
- nextRecord = readerContext.getDeleteRow(isDeleteAndRecord.getRight(),
baseRecordInfo.getRecordKey());
- readStats.incrementNumDeletes();
- return nextRecord != null;
- } else {
- // not emit Deletes
- readStats.incrementNumDeletes();
- return false;
- }
- }
+ protected boolean hasNextBaseRecord(T baseRecord) throws IOException {
+ BufferedRecord<T> logRecordInfo;
+ if (records.isEmpty()
+ || (logRecordInfo =
records.remove(readerContext.getRecordKey(baseRecord, readerSchema))) == null) {
+ // Inserts
+ nextRecord = getRecordForIterator(readerContext.seal(baseRecord), false);
Review Comment:
Make sense, I'll update it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
