danny0405 commented on code in PR #13115:
URL: https://github.com/apache/hudi/pull/13115#discussion_r2038544622
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/FileGroupRecordBuffer.java:
##########
@@ -249,14 +250,12 @@ protected Option<Pair<Option<T>, Map<String, Object>>>
doProcessNextDataRecord(T
// TODO(HUDI-7843): decouple the merging logic from the merger
// and use the record merge mode to control how to merge partial
updates
// Merge and store the combined record
- // Note that the incoming `record` is from an older commit, so it
should be put as
- // the `older` in the merge API
Option<Pair<HoodieRecord, Schema>> combinedRecordAndSchemaOpt =
recordMerger.get().partialMerge(
- readerContext.constructHoodieRecord(Option.of(record), metadata),
- readerContext.getSchemaFromMetadata(metadata),
readerContext.constructHoodieRecord(
existingRecordMetadataPair.getLeft(),
existingRecordMetadataPair.getRight()),
readerContext.getSchemaFromMetadata(existingRecordMetadataPair.getRight()),
+ readerContext.constructHoodieRecord(Option.of(record), metadata),
+ readerContext.getSchemaFromMetadata(metadata),
readerSchema,
props);
if (!combinedRecordAndSchemaOpt.isPresent()) {
Review Comment:
> its ok to always return value from doProcessNextDataRecord and update the
map w/ the same record as prev version.
not really, for spillable, less put/get means less SE/DE of the records.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
