[
https://issues.apache.org/jira/browse/AVRO-3106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael LeGore updated AVRO-3106:
---------------------------------
Description:
The Java JsonDecoder does not actually allow there to be unknown fields in the
reader compared to the writer's schema. In cases where this happens it gives
the following error:
{code:java}
org.apache.avro.AvroTypeException: Expected Unknown fields: [<field name
redacted>]. Got END_OBJECT
{code}
This seems to contradict Avro's [schema resolution
documentation|https://avro.apache.org/docs/current/spec.html#Schema+Resolution]
which states (emphasis mine)
{quote}if both are records:
* the ordering of fields may be different: fields are matched by name.
* schemas for fields with the same name in both records are resolved
recursively.
* *if the writer's record contains a field with a name not present in the
reader's record, the writer's value for that field is ignored.*
* if the reader's record schema has a field that contains a default value, and
writer's schema does not have a field with the same name, then the reader
should use the default value from its field.
* if the reader's record schema has a field with no default value, and
writer's schema does not have a field with the same name, an error is
signalled.{quote}
Looking at the codebase it seems to be happening at this line here:
[https://github.com/apache/avro/blob/589a9be3387aea2ad3c356964256eaff7645c3c2/lang/java/avro/src/main/java/org/apache/avro/io/JsonDecoder.java#L495]
was:
The Java JsonDecoder does not actually allow there to be unknown fields in the
reader compared to the writer's schema. In cases where this happens it gives
the following error:
org.apache.avro.AvroTypeException: Expected Unknown fields: [<field name
redacted>]. Got END_OBJECT
This seems to contradict Avro's [schema resolution
documentation|https://avro.apache.org/docs/current/spec.html#Schema+Resolution]
which states (emphasis mine)
{quote}if both are records:
* the ordering of fields may be different: fields are matched by name.
* schemas for fields with the same name in both records are resolved
recursively.
* *if the writer's record contains a field with a name not present in the
reader's record, the writer's value for that field is ignored.*
* if the reader's record schema has a field that contains a default value, and
writer's schema does not have a field with the same name, then the reader
should use the default value from its field.
* if the reader's record schema has a field with no default value, and
writer's schema does not have a field with the same name, an error is
signalled.{quote}
Looking at the codebase it seems to be happening at this line here:
[https://github.com/apache/avro/blob/589a9be3387aea2ad3c356964256eaff7645c3c2/lang/java/avro/src/main/java/org/apache/avro/io/JsonDecoder.java#L495]
> Java JsonDecoder does not allow unknown fields in reader
> --------------------------------------------------------
>
> Key: AVRO-3106
> URL: https://issues.apache.org/jira/browse/AVRO-3106
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.10.2
> Reporter: Michael LeGore
> Priority: Minor
>
> The Java JsonDecoder does not actually allow there to be unknown fields in
> the reader compared to the writer's schema. In cases where this happens it
> gives the following error:
> {code:java}
> org.apache.avro.AvroTypeException: Expected Unknown fields: [<field name
> redacted>]. Got END_OBJECT
> {code}
> This seems to contradict Avro's [schema resolution
> documentation|https://avro.apache.org/docs/current/spec.html#Schema+Resolution]
> which states (emphasis mine)
> {quote}if both are records:
> * the ordering of fields may be different: fields are matched by name.
> * schemas for fields with the same name in both records are resolved
> recursively.
> * *if the writer's record contains a field with a name not present in the
> reader's record, the writer's value for that field is ignored.*
> * if the reader's record schema has a field that contains a default value,
> and writer's schema does not have a field with the same name, then the reader
> should use the default value from its field.
> * if the reader's record schema has a field with no default value, and
> writer's schema does not have a field with the same name, an error is
> signalled.{quote}
> Looking at the codebase it seems to be happening at this line here:
> [https://github.com/apache/avro/blob/589a9be3387aea2ad3c356964256eaff7645c3c2/lang/java/avro/src/main/java/org/apache/avro/io/JsonDecoder.java#L495]
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
