pratyakshsharma commented on a change in pull request #1427: [HUDI-727]: Copy
default values of fields if not present when rewriting incoming record with new
schema
URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r403739002
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/HoodieAvroUtils.java
##########
@@ -191,21 +191,25 @@ public static GenericRecord
addCommitMetadataToRecord(GenericRecord record, Stri
* schema.
*/
public static GenericRecord rewriteRecord(GenericRecord record, Schema
newSchema) {
- return rewrite(record, record.getSchema(), newSchema);
+ return rewrite(record, getAllFieldsToWrite(record.getSchema(), newSchema),
newSchema);
}
/**
* Given a avro record with a given schema, rewrites it into the new schema
while setting fields only from the new
* schema.
*/
public static GenericRecord
rewriteRecordWithOnlyNewSchemaFields(GenericRecord record, Schema newSchema) {
- return rewrite(record, newSchema, newSchema);
+ return rewrite(record, newSchema.getFields(), newSchema);
}
- private static GenericRecord rewrite(GenericRecord record, Schema
schemaWithFields, Schema newSchema) {
+ private static GenericRecord rewrite(GenericRecord record, List<Field>
fieldsToWrite, Schema newSchema) {
GenericRecord newRecord = new GenericData.Record(newSchema);
- for (Schema.Field f : schemaWithFields.getFields()) {
- newRecord.put(f.name(), record.get(f.name()));
+ for (Schema.Field f : fieldsToWrite) {
+ if (record.get(f.name()) == null) {
Review comment:
Actually in avro, actual field value is maintained with the
GenericData.Record class but defaultValue is maintained with Schema.Field. So I
guess Avro expects users to fetch them separately.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
