You need to specify both the reader and writer schema in the
SpecificDatumReader. The writer schema is used in serialize and the reader
schema is the schema you want to use after deserialize. See
http://avro.apache.org/docs/1.7.0/api/java/org/apache/avro/specific/SpecificDatumReader.html
Hope this helps.Sam
On Wednesday, July 29, 2015 7:26 AM, Yosi Botzer <[email protected]>
wrote:
Hi,
I have 2 schemas:
{"namespace": "com.slg.model.generated.event", "type": "record", "name": "A1",
"fields": [ {"name": "event_id", "type": "string", "doc" : "unique event id"},
{"name": "ts", "type": "long", "doc" : "time stamp in miliseconds"} ]}
and
{"namespace": "com.slg.model.generated.event", "type": "record", "name": "A2",
"fields": [ {"name": "event_id", "type": "string", "doc" : "unique event id"},
{"name": "ts", "type": "long", "doc" : "time stamp in miliseconds"}, {"name":
"msg", "type": ["null", "string"], "default" : null, "doc" : "error message"} ]}
As you can see there are identical except the A2 has one extra field with
default value.
When running the following code:
package common.events;
import java.io.ByteArrayOutputStream;import java.io.IOException;
import org.apache.avro.Schema;import org.apache.avro.io.BinaryEncoder;import
org.apache.avro.io.DatumWriter;import org.apache.avro.io.DecoderFactory;import
org.apache.avro.io.EncoderFactory;import
org.apache.avro.specific.SpecificDatumReader;import
org.apache.avro.specific.SpecificDatumWriter;import
org.apache.avro.specific.SpecificRecordBase;
import com.slg.model.generated.event.A1;import com.slg.model.generated.event.A2;
public class AddFieldTest {
public static void main(String[] args) throws IOException{ A1 a1 = new
A1("aaa1", 1000L); byte[] bytes = serialize(a1); A2 a2 =
deserialize(A2.SCHEMA$, bytes); System.out.println(a2); } public static <T> T
deserialize(Schema schema, byte[] value) throws IOException {
SpecificDatumReader<T> reader = new SpecificDatumReader<T>(schema); T record =
reader.read(null, DecoderFactory.get().binaryDecoder(value, null)); return
record; } public static <T extends SpecificRecordBase> byte[] serialize(T
record) throws IOException { try (ByteArrayOutputStream out = new
ByteArrayOutputStream()) { BinaryEncoder encoder =
EncoderFactory.get().binaryEncoder(out, null); DatumWriter<T> writer = new
SpecificDatumWriter<T>(record.getSchema()); writer.write(record, encoder);
encoder.flush(); return out.toByteArray(); } }}
I was surprised to get the following exception:
Exception in thread "main" java.io.EOFException at
org.apache.avro.io.BinaryDecoder.ensureBounds(BinaryDecoder.java:473) at
org.apache.avro.io.BinaryDecoder.readInt(BinaryDecoder.java:128) at
org.apache.avro.io.BinaryDecoder.readIndex(BinaryDecoder.java:423) at
org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:290) at
org.apache.avro.io.parsing.Parser.advance(Parser.java:88) at
org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:267) at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:155) at
org.apache.avro.generic.GenericDatumReader.readField(GenericDatumReader.java:193)
at
org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:183)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:151) at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142) at
common.events.AddFieldTest.deserialize(AddFieldTest.java:32) at
common.events.AddFieldTest.main(AddFieldTest.java:25)
Why can't the decoder populate the value of A2 with the default value?
Am I doing something wrong here?
Yosi
