[
https://issues.apache.org/jira/browse/AVRO-2160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17555063#comment-17555063
]
Oscar Westra van Holthe - Kind commented on AVRO-2160:
------------------------------------------------------
The tricky bit here is that with common JSON usage, missing fields yield a null
value upon evaluation. This is identical to an attribute with an explicit null
value.
But in Avro, "null" and "missing" are different cases. For the field
{{{}{"name":"lastname","type":["string","null"],"default":"last"}{}}}, a null
value from JSON evaluation should either yield null (if explicit), or "last"
(if implicit).
As JSON data contains an implicit schema, the JSON decoder could add default
values (from the write schema!) for missing attributes. But this does make
usage of the binary and JSON decoders incompatible with each other. So for
compatibility, all Avro decoders require all fields to be available.
> Json to Avro with non required value and union schema failing
> -------------------------------------------------------------
>
> Key: AVRO-2160
> URL: https://issues.apache.org/jira/browse/AVRO-2160
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.8.2
> Reporter: Lydie
> Priority: Critical
> Labels: java
>
> I am trying to convert this string:
> str str4
> using this schema:
> {"type":"record",
> "namespace":"foo","name":"Person","fields":[\\{"name":"lastname","type":
> ["null","string"], "default":null}
> ,\{"name":"firstname","type":"string"},{"name":"age","type":["null","int"],
> "default":null}]}
> I get this error
> {color:#ff0000}com.syapse.messagePublisher.publisher.AvroEncodeException:
> Expected field name not found:
> lastnamein\{"firstname":"John","age":{"int":35}}{color}at
> com.syapse.messagePublisher.publisher.AvroEncoder.convertJsonToAvro(AvroEncoder.java:78)
>
> Although this should give me the correct syntax for a non required filed.
> Note that it works for
> {"lastname":\\{"string" : "Doe"}
> ,"firstname":"John","age":\{"int":36}}
>
> What am I missing ( using Abro 1.8.2)
> here is my code:
>
> {code:java}
> public static byte[] convertJsonToAvro(byte[] data, String schemaStr) throws
> AvroEncodeException {
> InputStream input = null;
> DataFileWriter<GenericRecord> writer = null;
> ByteArrayOutputStream output = null;
> try {
> Schema schema = new Schema.Parser().parse(schemaStr);
> DatumReader<GenericRecord> reader = new
> GenericDatumReader<GenericRecord>(schema);
> input = new ByteArrayInputStream(data);
> DataInputStream din = new DataInputStream(input);
> output = new ByteArrayOutputStream();
> writer = new DataFileWriter<GenericRecord>(new
> GenericDatumWriter<GenericRecord>());
> writer.create(schema, output);
> Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
> GenericRecord datum = null;
> while (true) {
> try {
> datum = reader.read(null, decoder);
> } catch (EOFException eofe) {
> break;
> }
> writer.append(datum);
> }
> writer.flush();
> writer.close();
> return output.toByteArray();
> } catch (AvroTypeException e) {
> throw new AvroEncodeException(e.getMessage() + "in" + new String(data));
> } catch (IOException e1) {
> throw new AvroEncodeException("Error decoding Json " + e1.getMessage());
> } finally {
> try {
> input.close();
> } catch (Exception e) {
> }
> }
> }
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
