vinothchandar commented on a change in pull request #1427: [HUDI-727]: Copy
default values of fields if not present when rewriting incoming record with new
schema
URL: https://github.com/apache/incubator-hudi/pull/1427#discussion_r400422483
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/util/HoodieAvroUtils.java
##########
@@ -191,21 +191,25 @@ public static GenericRecord
addCommitMetadataToRecord(GenericRecord record, Stri
* schema.
*/
public static GenericRecord rewriteRecord(GenericRecord record, Schema
newSchema) {
- return rewrite(record, record.getSchema(), newSchema);
+ return rewrite(record, getAllFieldsToWrite(record.getSchema(), newSchema),
newSchema);
}
/**
* Given a avro record with a given schema, rewrites it into the new schema
while setting fields only from the new
* schema.
*/
public static GenericRecord
rewriteRecordWithOnlyNewSchemaFields(GenericRecord record, Schema newSchema) {
- return rewrite(record, newSchema, newSchema);
+ return rewrite(record, newSchema.getFields(), newSchema);
}
- private static GenericRecord rewrite(GenericRecord record, Schema
schemaWithFields, Schema newSchema) {
+ private static GenericRecord rewrite(GenericRecord record, List<Field>
fieldsToWrite, Schema newSchema) {
GenericRecord newRecord = new GenericData.Record(newSchema);
- for (Schema.Field f : schemaWithFields.getFields()) {
- newRecord.put(f.name(), record.get(f.name()));
+ for (Schema.Field f : fieldsToWrite) {
+ if (record.get(f.name()) == null) {
Review comment:
for my understanding, I would expect the default value to be handed to us
here, already right?
Is this a avro bug/quirk? How exactly does one use the default value if
get() won't hand it. avro expects users to manually fetch it from the schema?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
