Still troubleshooting this and I noticed I get the same exception even if I skip the Json encoding and just execute:
GenericData.get().validate(sinkFieldSchema, override) Where “sinkFieldSchema" is the schema of the field within the evolved persistence schema which corresponds to the message schema and “override” is the value of the message. I wouldn’t think validate() should throw. Is it a bug? More importantly, is there an alternative way to insert sub records into a super record that accounts for schema evolution? Thanks, Jake From: Jacob Maes <[email protected]<mailto:[email protected]>> Date: Tuesday, March 24, 2015 at 6:32 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: ArrayIndexOutOfBoundsException while writing an old sub record to an evolved superrecord Hello, I’m currently working on a system that receives Avro messages via Kafka which contain message records and then writes them to a persistent store using a schema that is a super record (contains all the schemas of the message records). Recently, I have had to evolve the persistence schema to include a couple extra (optional) fields. After making this schema change, I get an ArrayIndexOutOfBoundsException when the system receives an old version of the message and tries to inject it into the new version of the schema. I’m attaching a test case that demonstrates the issue. The testAvroNormalUpdate() case passes while the testAvroEvolvedUpdate() case produces the stack trace below. The new schema should be backward compatible, so I’m not sure where I went wrong. I’m using Avro 1.4, but I’ve confirmed the same issue on Avro 1.7.7. Any help would be much appreciated. Thanks, Jake java.lang.ArrayIndexOutOfBoundsException: 2 at org.apache.avro.generic.GenericData$Record.get(GenericData.java:64) at org.apache.avro.generic.GenericDatumWriter.getField(GenericDatumWriter.java:113) at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:100) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:64) at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:133) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:71) at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:102) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:64) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:56) at com.pub.avro.TestEvolvedSchema.genericRecordToJson(TestEvolvedSchema.java:234) at com.pub.avro.TestEvolvedSchema.genericRecordToJson(TestEvolvedSchema.java:240) at com.pub.avro.TestEvolvedSchema.testAvroEvolvedUpdate(TestEvolvedSchema.java:190)
