[jira] [Created] (AVRO-2485) ClassCastException on Nullable Map Value Schemas with Backward Compatible Changes (Field Addition)

[Sage Pierce (JIRA)]

Sage Pierce created AVRO-2485:
---------------------------------

             Summary: ClassCastException on Nullable Map Value Schemas with 
Backward Compatible Changes (Field Addition)
                 Key: AVRO-2485
                 URL: https://issues.apache.org/jira/browse/AVRO-2485
             Project: Apache Avro
          Issue Type: Bug
          Components: java
    Affects Versions: 1.9.0
         Environment: Operating System: Mac OS 10.13.6

JDK version: 1.8_172

JUnit version: 4.12
            Reporter: Sage Pierce


In one of our Java applications, upon upgrading from 
{{org.apache.avro:avro:1.8.2}} to {{org.apache.avro:avro:1.9.0}}, we are seeing 
deserialization failures with the following exception:
{code:java}
java.lang.ClassCastException: org.apache.avro.Resolver$ReaderUnion cannot be 
cast to org.apache.avro.Resolver$Container

        at 
org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:99)
        at 
org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:110)
        at 
org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:141)
        at 
org.apache.avro.io.parsing.ResolvingGrammarGenerator.generate(ResolvingGrammarGenerator.java:65)
        at org.apache.avro.io.ResolvingDecoder.resolve(ResolvingDecoder.java:85)
        at org.apache.avro.io.ResolvingDecoder.<init>(ResolvingDecoder.java:50)
        at 
org.apache.avro.io.DecoderFactory.resolvingDecoder(DecoderFactory.java:297)
        at 
org.apache.avro.generic.GenericDatumReader.getResolver(GenericDatumReader.java:128)
        at 
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:142)
{code}
Further inspection of the stack trace and problematic data shows the issue lies 
with nullable (unioned) Map Schemas, whose values are complex types, and those 
types having backward compatible changes between the Writer and Reader Schemas. 
The following test exposes the issue:
{code:java|title=AvroUpgradeTest.java}
import java.io.ByteArrayOutputStream;
import java.util.Collections;

import org.apache.avro.Schema;
import org.apache.avro.io.DecoderFactory;
import org.apache.avro.io.EncoderFactory;
import org.apache.avro.reflect.ReflectData;
import org.apache.avro.reflect.ReflectDatumWriter;
import org.junit.Test;

import static org.junit.Assert.assertEquals;

public class AvroUpgradeTest {

    @Test
    public void 
nullableMapWithBackwardCompatibleValueChangesCanBeDeserialized() throws 
Exception {
    TestDataWithNullableMap data = new TestDataWithNullableMap();
    data.setMap(Collections.singletonMap("KEY", TestData.create()));

    Schema writerSchema = 
ReflectData.get().getSchema(TestDataWithNullableMap.class);

    Schema readerMapSchema = 
Schema.createMap(Schema.createRecord(TestData.class.getName(), null, null, 
false,
        Collections.singletonList(new Schema.Field("data1", 
Schema.create(Schema.Type.STRING), null, Object.class.cast(null)))));
    Schema readerSchema = 
Schema.createRecord(TestDataWithNullableMap.class.getName(), null, null, false,
        Collections.singletonList(new Schema.Field("map", 
ReflectData.makeNullable(readerMapSchema), null, Object.class.cast(null))));

    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
    new ReflectDatumWriter<>(writerSchema, ReflectData.get()).write(data, 
EncoderFactory.get().directBinaryEncoder(outputStream, null));

    byte[] serialized = outputStream.toByteArray();

    TestDataWithNullableMap deserialized = new 
ReflectDecoderAvroDeserializer.ReflectDecoderDatumReader<TestDataWithNullableMap>(writerSchema,
 readerSchema, ReflectData.get())
        .read(null, DecoderFactory.get().binaryDecoder(serialized, 0, 
serialized.length, null));

    assertEquals(data.getMap().get("KEY").getData1(), 
deserialized.getMap().get("KEY").getData1());
    }
}
{code}
Note: I'm having to do a some manual Schema generation to force the appearance 
of the Writer's Schema making a (backward compatible) field addition.

Relevant POJOs are as follows:
{code:java|title=TestDataWithNullableMap.java}
import java.util.Map;

import org.apache.avro.reflect.Nullable;

public class TestDataWithNullableMap {

    @Nullable
    private Map<String, TestData> map;

    public Map<String, TestData> getMap() {
        return map;
    }

    public void setMap(Map<String, TestData> map) {
        this.map = map;
    }
}
{code}
—
{code:java|title=TestData.java}
public class TestData {

    private String data1;

    private String data2;

    public static TestData create() {
        TestData data = new TestData();
        data.setData1("DATA 1");
        data.setData2("DATA 2");
        return data;
    }

    public String getData1() {
        return data1;
    }

    public void setData1(String data1) {
        this.data1 = data1;
    }

    public String getData2() {
        return data2;
    }

    public void setData2(String data2) {
        this.data2 = data2;
    }
}
{code}
 

The preceding test succeeds with Avro 1.8.2, but fails on Avro 1.9.0 with the 
mentioned Exception.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)