[
https://issues.apache.org/jira/browse/HUDI-7152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo updated HUDI-7152:
----------------------------
Description:
The default value can be in the following format in the JSON schema: (1)
{{{}"field":\{"type":"integer","default":"50"}{}}}, (2)
{{{}"field":\{"oneOf":[{"type":"null"},\{"type":"integer","default":"60"}]}{}}}.
In both cases, while the field type is integer (non-String), the default value
is put as a String value. The Avro schema requires strict type match for the
default value and correct ordering of types (e.g.,
{{{}"type":["long","null"]{}}}). Right now the Avro schema with the default
value generated is not valid, causing the following exception
{code:java}
Caused by: org.apache.avro.AvroTypeException: Invalid default for field field1:
"0" not a ["null","long"]
at org.apache.avro.Schema.validateDefault(Schema.java:1576)
at org.apache.avro.Schema.access$500(Schema.java:91)
at org.apache.avro.Schema$Field.<init>(Schema.java:549)
at org.apache.avro.Schema.parse(Schema.java:1683)
at org.apache.avro.Schema.parse(Schema.java:1760)
at org.apache.avro.Schema.parse(Schema.java:1673)
at org.apache.avro.Schema$Parser.parse(Schema.java:1430)
at org.apache.avro.Schema$Parser.parse(Schema.java:1418)
at
org.apache.hudi.utilities.schema.SchemaRegistryProvider.parseSchemaFromRegistry(SchemaRegistryProvider.java:79)
... 14 more {code}
> Fix default value parsing in JSON schema converter
> --------------------------------------------------
>
> Key: HUDI-7152
> URL: https://issues.apache.org/jira/browse/HUDI-7152
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Priority: Major
> Fix For: 0.14.1
>
>
> The default value can be in the following format in the JSON schema: (1)
> {{{}"field":\{"type":"integer","default":"50"}{}}}, (2)
> {{{}"field":\{"oneOf":[{"type":"null"},\{"type":"integer","default":"60"}]}{}}}.
> In both cases, while the field type is integer (non-String), the default
> value is put as a String value. The Avro schema requires strict type match
> for the default value and correct ordering of types (e.g.,
> {{{}"type":["long","null"]{}}}). Right now the Avro schema with the default
> value generated is not valid, causing the following exception
> {code:java}
> Caused by: org.apache.avro.AvroTypeException: Invalid default for field
> field1: "0" not a ["null","long"]
> at org.apache.avro.Schema.validateDefault(Schema.java:1576)
> at org.apache.avro.Schema.access$500(Schema.java:91)
> at org.apache.avro.Schema$Field.<init>(Schema.java:549)
> at org.apache.avro.Schema.parse(Schema.java:1683)
> at org.apache.avro.Schema.parse(Schema.java:1760)
> at org.apache.avro.Schema.parse(Schema.java:1673)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1430)
> at org.apache.avro.Schema$Parser.parse(Schema.java:1418)
> at
> org.apache.hudi.utilities.schema.SchemaRegistryProvider.parseSchemaFromRegistry(SchemaRegistryProvider.java:79)
> ... 14 more {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
