[
https://issues.apache.org/jira/browse/HUDI-8521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17899566#comment-17899566
]
Y Ethan Guo commented on HUDI-8521:
-----------------------------------
Also we need to check 0.12, 0.14 and 0.15 release. It looks like overwrite
with latest can generate different results with compaction in MOR in 0.14 and
0.15 in terms of the expected semantics of commit time ordering. But it
somehow is consistent with old log scanner logic using preCombine.
HoodieMergedLogRecordScanner in Hudi 0.12:
{code:java}
@Override
protected void processNextRecord(HoodieRecord<? extends HoodieRecordPayload>
hoodieRecord) throws IOException {
String key = hoodieRecord.getRecordKey();
if (records.containsKey(key)) {
// Merge and store the merged record. The HoodieRecordPayload
implementation is free to decide what should be
// done when a DELETE (empty payload) is encountered before or after an
insert/update.
HoodieRecord<? extends HoodieRecordPayload> oldRecord = records.get(key);
HoodieRecordPayload oldValue = oldRecord.getData();
HoodieRecordPayload combinedValue =
hoodieRecord.getData().preCombine(oldValue);
// If combinedValue is oldValue, no need rePut oldRecord
if (combinedValue != oldValue) {
HoodieOperation operation = hoodieRecord.getOperation();
records.put(key, new HoodieAvroRecord<>(new HoodieKey(key,
hoodieRecord.getPartitionPath()), combinedValue, operation));
}
} else {
// Put the record as is
records.put(key, hoodieRecord);
}
} {code}
> Resolve issues w/ diff merge modes
> -----------------------------------
>
> Key: HUDI-8521
> URL: https://issues.apache.org/jira/browse/HUDI-8521
> Project: Apache Hudi
> Issue Type: Bug
> Components: reader-core, writer-core
> Reporter: sivabalan narayanan
> Assignee: Lin Liu
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.0.0
>
>
> we found some issues w/ merge mode feature in 1.x.
> we need to triage them and fix
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
