xiarixiaoyao commented on a change in pull request #3181:
URL: https://github.com/apache/hudi/pull/3181#discussion_r664316683
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieFileSliceReader.java
##########
@@ -41,11 +43,26 @@
while (baseIterator.hasNext()) {
GenericRecord record = (GenericRecord) baseIterator.next();
HoodieRecord<T> hoodieRecord =
SpillableMapUtils.convertToHoodieRecordPayload(record, payloadClass);
- scanner.processNextRecord(hoodieRecord);
+ processNextRecord(scanner, hoodieRecord);
}
return new HoodieFileSliceReader(scanner.iterator());
}
+ private static void processNextRecord(HoodieMergedLogRecordScanner scanner,
HoodieRecord<? extends HoodieRecordPayload> hoodieRecord) {
Review comment:
I'm afraid it will lead some complex changes
1）all implementation classes for HoodieRecordPayload has not implemented
preCombine(T another, Properties properties)
2) we need modfiy preCombine(T another, Properties properties) to
preCombine(T another, Properties properties, Scheme schema), since we need to
get the value of HoodiePayloadProps.PAYLOAD_ORDERING_FIELD_PROP from current
payload to do compare with another payload. Obviously, we can pass in a
special value to determine which value to keep， but i think this is a tricky。
of course， If you insist， i will update code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
