[
https://issues.apache.org/jira/browse/AVRO-2485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16892879#comment-16892879
]
Nandor Kollar commented on AVRO-2485:
-------------------------------------
[~wisegas] I think this is a know issue (see AVRO-2400 and AVRO-2415), which is
fixed on master, and will be released in 1.9.1.
> ClassCastException on Nullable Map Value Schemas with Backward Compatible
> Changes (Field Addition)
> --------------------------------------------------------------------------------------------------
>
> Key: AVRO-2485
> URL: https://issues.apache.org/jira/browse/AVRO-2485
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.9.0
> Environment: Operating System: Mac OS 10.13.6
> JDK version: 1.8_172
> JUnit version: 4.12
> Reporter: Sage Pierce
> Priority: Critical
>
> In one of our Java applications, upon upgrading from
> {{org.apache.avro:avro:1.8.2}} to {{org.apache.avro:avro:1.9.0}}, we are
> seeing deserialization failures with the following exception:
> {code:java}
> java.lang.ClassCastException: org.apache.avro.Resolver$ReaderUnion cannot be
> cast to org.apache.avro.Resolver$Container
> at
> org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:99)
> at
> org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:110)
> at
> org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:141)
> at
> org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:65)
> at org.apache.avro.io.ResolvingDecoder.resolve(ResolvingDecoder.java:85)
> at org.apache.avro.io.ResolvingDecoder.<init>(ResolvingDecoder.java:50)
> at
> org.apache.avro.io.DecoderFactory.resolvingDecoder(DecoderFactory.java:297)
> at
> org.apache.avro.generic.GenericDatumReader.getResolver(GenericDatumReader.java:128)
> at
> org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
> {code}
> Further inspection of the stack trace and problematic data shows the issue
> lies with nullable (unioned) Map Schemas, whose values are complex types, and
> those types having backward compatible changes between the Writer and Reader
> Schemas. The following test exposes the issue:
> {code:java|title=AvroUpgradeTest.java}
> import java.io.ByteArrayOutputStream;
> import java.util.Collections;
> import org.apache.avro.Schema;
> import org.apache.avro.io.DecoderFactory;
> import org.apache.avro.io.EncoderFactory;
> import org.apache.avro.reflect.ReflectData;
> import org.apache.avro.reflect.ReflectDatumWriter;
> import org.junit.Test;
> import static org.junit.Assert.assertEquals;
> public class AvroUpgradeTest {
> @Test
> public void
> nullableMapWithBackwardCompatibleValueChangesCanBeDeserialized() throws
> Exception {
> TestDataWithNullableMap data = new TestDataWithNullableMap();
> data.setMap(Collections.singletonMap("KEY", TestData.create()));
> Schema writerSchema =
> ReflectData.get().getSchema(TestDataWithNullableMap.class);
> Schema readerMapSchema =
> Schema.createMap(Schema.createRecord(TestData.class.getName(), null, null,
> false,
> Collections.singletonList(new Schema.Field("data1",
> Schema.create(Schema.Type.STRING), null, Object.class.cast(null)))));
> Schema readerSchema =
> Schema.createRecord(TestDataWithNullableMap.class.getName(), null, null,
> false,
> Collections.singletonList(new Schema.Field("map",
> ReflectData.makeNullable(readerMapSchema), null, Object.class.cast(null))));
> ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
> new ReflectDatumWriter<>(writerSchema, ReflectData.get()).write(data,
> EncoderFactory.get().directBinaryEncoder(outputStream, null));
> byte[] serialized = outputStream.toByteArray();
> TestDataWithNullableMap deserialized = new
> ReflectDecoderAvroDeserializer.ReflectDecoderDatumReader<TestDataWithNullableMap>(writerSchema,
> readerSchema, ReflectData.get())
> .read(null, DecoderFactory.get().binaryDecoder(serialized, 0,
> serialized.length, null));
> assertEquals(data.getMap().get("KEY").getData1(),
> deserialized.getMap().get("KEY").getData1());
> }
> }
> {code}
> Note: I'm having to do a some manual Schema generation to force the
> appearance of the Writer's Schema making a (backward compatible) field
> addition.
> Relevant POJOs are as follows:
> {code:java|title=TestDataWithNullableMap.java}
> import java.util.Map;
> import org.apache.avro.reflect.Nullable;
> public class TestDataWithNullableMap {
> @Nullable
> private Map<String, TestData> map;
> public Map<String, TestData> getMap() {
> return map;
> }
> public void setMap(Map<String, TestData> map) {
> this.map = map;
> }
> }
> {code}
> —
> {code:java|title=TestData.java}
> public class TestData {
> private String data1;
> private String data2;
> public static TestData create() {
> TestData data = new TestData();
> data.setData1("DATA 1");
> data.setData2("DATA 2");
> return data;
> }
> public String getData1() {
> return data1;
> }
> public void setData1(String data1) {
> this.data1 = data1;
> }
> public String getData2() {
> return data2;
> }
> public void setData2(String data2) {
> this.data2 = data2;
> }
> }
> {code}
>
> The preceding test succeeds with Avro 1.8.2, but fails on Avro 1.9.0 with the
> mentioned Exception.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
