cshuo commented on code in PR #13408:
URL: https://github.com/apache/hudi/pull/13408#discussion_r2136752800
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java:
##########
@@ -376,14 +377,27 @@ public Option<IndexedRecord>
combineAndGetUpdateValue(IndexedRecord oldRecord, S
}
@Override
- public Option<IndexedRecord> getInsertValue(Schema schemaIgnored, Properties
propertiesIgnored) throws IOException {
+ public Option<IndexedRecord> getInsertValue(Schema schema, Properties
propertiesIgnored) throws IOException {
if (key == null || this.isDeletedRecord) {
return Option.empty();
}
- HoodieMetadataRecord record = new HoodieMetadataRecord(key, type,
filesystemMetadata, bloomFilterMetadata,
- columnStatMetadata, recordIndexMetadata, secondaryIndexMetadata);
- return Option.of(record);
+ if (schema != null &&
schema.getField(HoodieRecord.COMMIT_TIME_METADATA_FIELD) != null) {
+ GenericRecord result = new GenericData.Record(schema);
Review Comment:
* `HoodieMetadataPayload#getInsertValue` will be called as the merging
result during FG reader merging metadata record.
* Currently `HoodieMetadataPayload#getInsertValue` always return fixed
schema containing 7 data fields, which may break the logics in File Group
reader, e.g., File Group reader has a required schema more than 7 data fields.
* The fix in this PR is using required schema to convert avro bytes back
into GenericRecord.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
