nsivabalan commented on code in PR #13115:
URL: https://github.com/apache/hudi/pull/13115#discussion_r2038339538
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/FileGroupRecordBuffer.java:
##########
@@ -249,14 +250,12 @@ protected Option<Pair<Option<T>, Map<String, Object>>>
doProcessNextDataRecord(T
// TODO(HUDI-7843): decouple the merging logic from the merger
// and use the record merge mode to control how to merge partial
updates
// Merge and store the combined record
- // Note that the incoming `record` is from an older commit, so it
should be put as
- // the `older` in the merge API
Option<Pair<HoodieRecord, Schema>> combinedRecordAndSchemaOpt =
recordMerger.get().partialMerge(
- readerContext.constructHoodieRecord(Option.of(record), metadata),
- readerContext.getSchemaFromMetadata(metadata),
readerContext.constructHoodieRecord(
existingRecordMetadataPair.getLeft(),
existingRecordMetadataPair.getRight()),
readerContext.getSchemaFromMetadata(existingRecordMetadataPair.getRight()),
+ readerContext.constructHoodieRecord(Option.of(record), metadata),
+ readerContext.getSchemaFromMetadata(metadata),
readerSchema,
props);
if (!combinedRecordAndSchemaOpt.isPresent()) {
Review Comment:
Also, thinking if we should standardize how a deleted record could be
represented in FG reader.
a deleted record from delete block, HoodieEmptyRecordPayload etc.
and if we have above abstraction in place (FileGroupRecord), we can also see
if we can avoid using DeleteRecord.
Its again causing some confusion since, doProcessNextDeletedRecord return
Option < DeleteRecord > , but while the caller processes is and updates the
`records` map which eventually results in the format Pair<Option, Map<String,
Object>>.
so, in the end, every record is represented in the same format in the cached
map of `records`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
