the-other-tim-brown commented on code in PR #13964:
URL: https://github.com/apache/hudi/pull/13964#discussion_r2372715530
##########
hudi-common/src/main/java/org/apache/hudi/avro/AvroRecordContext.java:
##########
@@ -136,6 +136,15 @@ public IndexedRecord extractDataFromRecord(HoodieRecord
record, Schema schema, P
}
}
+ @Override
+ public IndexedRecord extractDeflatedDataFromRecord(HoodieRecord record,
Schema schema, Properties properties) {
Review Comment:
Let's remove this based on our offline conversation
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroIndexedRecord.java:
##########
@@ -376,4 +378,8 @@ static void updateMetadataValuesInternal(GenericRecord
avroRecord, MetadataValue
public IndexedRecord getData() {
return binaryRecord.getRecord();
Review Comment:
This should return binaryRecord, check out my original draft
##########
hudi-common/src/main/java/org/apache/hudi/common/model/SerializableIndexedRecord.java:
##########
@@ -59,17 +62,17 @@ private SerializableIndexedRecord(IndexedRecord record) {
@Override
public void put(int i, Object v) {
- record.put(i, v);
+ getData().put(i, v);
}
@Override
public Object get(int i) {
- return record.get(i);
+ return getData().get(i);
}
@Override
public Schema getSchema() {
- return record.getSchema();
+ return getData().getSchema();
Review Comment:
We should update this to return `schema` if it is non-null and then
`record.getSchema()` otherwise so we don't eagerly deserialize just to return
the schema.
##########
hudi-common/src/main/java/org/apache/hudi/common/model/HoodieAvroIndexedRecord.java:
##########
@@ -376,4 +378,8 @@ static void updateMetadataValuesInternal(GenericRecord
avroRecord, MetadataValue
public IndexedRecord getData() {
return binaryRecord.getRecord();
}
+
+ public IndexedRecord getSerializableIndexedRecord() {
Review Comment:
Let's remove this and just use `getData`
##########
hudi-common/src/main/java/org/apache/hudi/common/table/read/BufferedRecords.java:
##########
@@ -48,6 +48,21 @@ public static <T> BufferedRecord<T>
fromHoodieRecord(HoodieRecord record, Schema
return new BufferedRecord<>(recordKey,
recordContext.convertOrderingValueToEngineType(orderingValue), data, schemaId,
inferOperation(isDelete, record.getOperation()));
}
+ public static <T> BufferedRecord<T>
fromHoodieRecordWithDeflatedRecord(HoodieRecord record, Schema schema,
RecordContext<T> recordContext, Properties props, String[] orderingFields) {
+ boolean isDelete = record.isDelete(schema, props);
+ return fromHoodieRecordWithDeflatedRecord(record, schema, recordContext,
props, orderingFields, isDelete);
+ }
+
+ public static <T> BufferedRecord<T>
fromHoodieRecordWithDeflatedRecord(HoodieRecord record, Schema schema,
RecordContext<T> recordContext,
+
Properties props, String[] orderingFields, boolean isDelete) {
+ HoodieKey hoodieKey = record.getKey();
+ T data = recordContext.extractDeflatedDataFromRecord(record, schema,
props);
+ String recordKey = hoodieKey == null ? recordContext.getRecordKey(data,
schema) : hoodieKey.getRecordKey();
+ Integer schemaId = recordContext.encodeAvroSchema(schema);
+ Comparable orderingValue = record.getOrderingValue(schema, props,
orderingFields);
+ return new BufferedRecord<>(recordKey,
recordContext.convertOrderingValueToEngineType(orderingValue), data, schemaId,
inferOperation(isDelete, record.getOperation()));
+ }
+
Review Comment:
This is no different from the `fromHoodieRecord` methods it seems once the
fixes are made that I mention above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
