danny0405 commented on code in PR #12285:
URL: https://github.com/apache/hudi/pull/12285#discussion_r1847535288
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieMergedLogRecordScanner.java:
##########
@@ -242,17 +245,25 @@ protected <T> void processNextRecord(HoodieRecord<T>
newRecord) throws IOExcepti
String key = newRecord.getRecordKey();
HoodieRecord<T> prevRecord = records.get(key);
if (prevRecord != null) {
- // Merge and store the combined record
- HoodieRecord<T> combinedRecord = (HoodieRecord<T>)
recordMerger.merge(prevRecord, readerSchema,
- newRecord, readerSchema, this.getPayloadProps()).get().getLeft();
- // If pre-combine returns existing record, no need to update it
- if (combinedRecord.getData() != prevRecord.getData()) {
- HoodieRecord latestHoodieRecord = getLatestHoodieRecord(newRecord,
combinedRecord, key);
-
- // NOTE: Record have to be cloned here to make sure if it holds
low-level engine-specific
- // payload pointing into a shared, mutable (underlying) buffer
we get a clean copy of
- // it since these records will be put into records(Map).
- records.put(key, latestHoodieRecord.copy());
+ switch (mergeMode) {
+ case OVERWRITE_WITH_LATEST:
Review Comment:
There are some responsibility overlapping with the specific payload class:
`OverriteWithLatestAvroPayload`, we should clarify how the merge
mode/merger/payload class cooperate here.
And should we call this `PROC_TIME_ORDERING` akka to `EVENT_TIME_ORDERING` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
