[
https://issues.apache.org/jira/browse/HUDI-727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bhavani Sudha updated HUDI-727:
-------------------------------
Fix Version/s: 0.5.3
> Copy default values of fields if not present when rewriting incoming record
> with new schema
> -------------------------------------------------------------------------------------------
>
> Key: HUDI-727
> URL: https://issues.apache.org/jira/browse/HUDI-727
> Project: Apache Hudi (incubating)
> Issue Type: Improvement
> Components: Utilities
> Reporter: Pratyaksh Sharma
> Assignee: Pratyaksh Sharma
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.6.0, 0.5.3
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently we recommend users to evolve schema in backwards compatible way.
> When one is trying to evolve schema in backwards compatible way, one of the
> most significant things to do is to define default value for newly added
> columns so that records published with previous schema also can be consumed
> properly.
>
> However just before actually writing record to Hudi dataset, we try to
> rewrite record with new Avro schema which has Hudi metadata columns [1]. In
> this function, we are only trying to get the values from record without
> considering field's default value. As a result, schema validation fails.
> IMO, this piece of code should take into account default value as well in
> case field's actual value is null.
>
> [1]
> [https://github.com/apache/incubator-hudi/blob/078d4825d909b2c469398f31c97d2290687321a8/hudi-common/src/main/java/org/apache/hudi/common/util/HoodieAvroUtils.java#L205].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
