[
https://issues.apache.org/jira/browse/AVRO-4012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oscar Caraballo updated AVRO-4012:
----------------------------------
Description:
*Given* the following Schema:
{code:java}
{
"type": "record",
"name": "User",
"namespace": "com.example",
"fields": [
{ "name": "name", "type": "string" }
,
{ "name": "email", "type": [ "null", "string" ], "default": null }
,
{
"name": "status",
"type":
{ "type": "enum", "name": "Status", "symbols": [ "ACTIVE", "INACTIVE",
"UNKNOWN" ], "default": "UNKNOWN" }
,
"default": "UNKNOWN"
}
]
} {code}
and Generating the avro classes with avro tools java -jar
./avro-tools-1.11.0.jar compile schema ./schema-v1.avsc .
and creating a json with a non existing enum value:
{code:java}
{"name":"John Doe","email":\{"string":"[email protected]"}
,"status":"PENDING"} {code}
*When* executing this test:
{code:java}
@Test
public void testJsonAgainstAvroSchemaDefault() throwsIOException
{
// Load Avro schema
Schemaschema=newParser().parse(getFilesStream("schema.avsc"));
// Load JSON file ObjectMappermapper=newObjectMapper();
JsonNodejsonNode=mapper.readTree(getFilesStream("example-2.json"));
// Convert JSON to Avro GenericRecord Userrecord=jsonToAvro(jsonNode, schema);
// Validate the record assertNotNull(record);
assertTrue(record.get("name").toString().equals("John Doe"));
assertTrue(record.get("email").toString().equals("[email protected]"));
assertTrue(record.get("status").toString().equals("UNKNOWN"));
} {code}
*Then* this error is signalled:
org.apache.avro.AvroTypeException: Unknown symbol in enum PENDING
at
org.apache.avro.io.JsonDecoder.readEnum([{*}JsonDecoder.java:331{*}|vscode-file://vscode-app/Applications...
But according to avro specification
[https://avro.apache.org/docs/1.11.1/specification/#schema-resolution]
_"{*}if both are enums{*}: if the writer’s symbol is not present in the
reader’s enum and the reader has a default value, then that value is used,
otherwise an error is signalled."_
JSONDecoder should evaluate if there is a default enum value available.
was:
*Given* the following Schema:
{
"type": "record",
"name": "User",
"namespace": "com.example",
"fields": [
{
"name": "name",
"type": "string"
},
{
"name": "email",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "status",
"type": {
"type": "enum",
"name": "Status",
"symbols": [
"ACTIVE",
"INACTIVE",
"UNKNOWN"
],
"default": "UNKNOWN"
},
"default": "UNKNOWN"
}
]
}
and Generating the avro classes with avro tools java -jar
./avro-tools-1.11.0.jar compile schema ./schema-v1.avsc .
and creating a json with a non existing enum value:
{"name":"John
Doe","email":\{"string":"[email protected]"},"status":"PENDING"}
*When* executing this test:
@Test
public void testJsonAgainstAvroSchemaDefault() throwsIOException {
// Load Avro schema
Schemaschema=newParser().parse(getFilesStream("schema.avsc"));
// Load JSON file
ObjectMappermapper=newObjectMapper();
JsonNodejsonNode=mapper.readTree(getFilesStream("example-2.json"));
// Convert JSON to Avro GenericRecord
Userrecord=jsonToAvro(jsonNode, schema);
// Validate the record
assertNotNull(record);
assertTrue(record.get("name").toString().equals("John Doe"));
assertTrue(record.get("email").toString().equals("[email protected]"));
assertTrue(record.get("status").toString().equals("UNKNOWN"));
}
*Then* this error is signalled:
org.apache.avro.AvroTypeException: Unknown symbol in enum PENDING
at
org.apache.avro.io.JsonDecoder.readEnum([JsonDecoder.java:331|vscode-file://vscode-app/Applications/Visual%20Studio%20Code.app/Contents/Resources/app/out/vs/code/electron-sandbox/workbench/workbench.html])
But according to avro specification
[https://avro.apache.org/docs/1.11.1/specification/#schema-resolution]
_"{*}if both are enums{*}: if the writer’s symbol is not present in the
reader’s enum and the reader has a default value, then that value is used,
otherwise an error is signalled."_
JSONDecoder should evaluate if there is a default enum value available.
> JSONDecoder do not evaluate if there is a default enum value available.
> -----------------------------------------------------------------------
>
> Key: AVRO-4012
> URL: https://issues.apache.org/jira/browse/AVRO-4012
> Project: Apache Avro
> Issue Type: Bug
> Affects Versions: 1.11.3
> Reporter: Oscar Caraballo
> Priority: Minor
>
> *Given* the following Schema:
>
> {code:java}
> {
> "type": "record",
> "name": "User",
> "namespace": "com.example",
> "fields": [
> { "name": "name", "type": "string" }
> ,
> { "name": "email", "type": [ "null", "string" ], "default": null }
> ,
> {
> "name": "status",
> "type":
> { "type": "enum", "name": "Status", "symbols": [ "ACTIVE", "INACTIVE",
> "UNKNOWN" ], "default": "UNKNOWN" }
> ,
> "default": "UNKNOWN"
> }
> ]
> } {code}
>
>
>
>
> and Generating the avro classes with avro tools java -jar
> ./avro-tools-1.11.0.jar compile schema ./schema-v1.avsc .
> and creating a json with a non existing enum value:
> {code:java}
> {"name":"John Doe","email":\{"string":"[email protected]"}
> ,"status":"PENDING"} {code}
>
> *When* executing this test:
>
> {code:java}
> @Test
> public void testJsonAgainstAvroSchemaDefault() throwsIOException
> {
> // Load Avro schema
> Schemaschema=newParser().parse(getFilesStream("schema.avsc"));
> // Load JSON file ObjectMappermapper=newObjectMapper();
> JsonNodejsonNode=mapper.readTree(getFilesStream("example-2.json"));
> // Convert JSON to Avro GenericRecord Userrecord=jsonToAvro(jsonNode, schema);
> // Validate the record assertNotNull(record);
> assertTrue(record.get("name").toString().equals("John Doe"));
> assertTrue(record.get("email").toString().equals("[email protected]"));
> assertTrue(record.get("status").toString().equals("UNKNOWN"));
>
> } {code}
>
> *Then* this error is signalled:
> org.apache.avro.AvroTypeException: Unknown symbol in enum PENDING
> at
> org.apache.avro.io.JsonDecoder.readEnum([{*}JsonDecoder.java:331{*}|vscode-file://vscode-app/Applications...
> But according to avro specification
> [https://avro.apache.org/docs/1.11.1/specification/#schema-resolution]
> _"{*}if both are enums{*}: if the writer’s symbol is not present in the
> reader’s enum and the reader has a default value, then that value is used,
> otherwise an error is signalled."_
> JSONDecoder should evaluate if there is a default enum value available.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
