alexeykudinkin commented on code in PR #6745:
URL: https://github.com/apache/hudi/pull/6745#discussion_r994933816
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroRecordMerger.java:
##########
@@ -67,12 +74,12 @@ private HoodieRecord preCombine(HoodieRecord older,
HoodieRecord newer) {
}
private Option<HoodieRecord> combineAndGetUpdateValue(HoodieRecord older,
HoodieRecord newer, Schema schema, Properties props) throws IOException {
- Option<HoodieAvroIndexedRecord> previousRecordAvroPayload =
older.toIndexedRecord(schema, props);
- if (!previousRecordAvroPayload.isPresent()) {
+ Option<IndexedRecord> previousAvroData = older.toIndexedRecord(schema,
props).map(HoodieAvroIndexedRecord::getData);
+ if (!previousAvroData.isPresent()) {
return Option.empty();
}
- return ((HoodieAvroRecord)
newer).getData().combineAndGetUpdateValue(previousRecordAvroPayload.get().getData(),
schema, props)
+ return ((HoodieAvroRecord)
newer).getData().combineAndGetUpdateValue(previousAvroData.get(), schema, props)
.map(combinedAvroPayload -> new
HoodieAvroIndexedRecord((IndexedRecord) combinedAvroPayload));
Review Comment:
You're right, i double-checked and we do indeed not shuffle the
`HoodieAvroIndexedRecord`
##########
hudi-common/src/main/java/org/apache/hudi/common/util/ParquetReaderIterator.java:
##########
@@ -36,24 +35,20 @@
private final ParquetReader<T> parquetReader;
// Holds the next entry returned by the parquet reader
private T next;
- // For directly use InternalRow
- private Function<T, T> mapper;
+ // Whether next is consumed
+ private boolean consumed;
Review Comment:
Why are we changing this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
