[
https://issues.apache.org/jira/browse/AVRO-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arseniy Tashoyan updated AVRO-3438:
-----------------------------------
Description:
We have an Apache Flink application, that processes Avro GenericRecords. Flink
uses Kryo serialization. The serialization fails:
com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
props (org.apache.avro.Schema$Field)
fieldMap (org.apache.avro.Schema$RecordSchema)
schema (org.apache.avro.generic.GenericData$Record)
values (org.apache.avro.generic.GenericData$Record)
values (org.apache.avro.generic.GenericData$Record)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:95)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
~[kryo-2.24.0.jar:?]
...
Caused by: java.lang.NullPointerException
at org.apache.avro.JsonProperties$2$1$1.<init>(JsonProperties.java:175)
~[avro-1.10.2.jar:1.10.2]
at org.apache.avro.JsonProperties$2$1.iterator(JsonProperties.java:174)
~[avro-1.10.2.jar:1.10.2]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:80)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
~[kryo-2.24.0.jar:?]
...
This problem occurs, because Kryo uses Unsafe to copy objects. When creating an
object, Unsafe does not call the class constructor. Therefore the field *props*
in the class JsonProperties is not properly initialized. The field *props* is
implemented via anonymous inner classes. The NullPointerException occurs at
JsonProperties.java:175, because the reference to the enclosing instance of
AbstractSet is null.
A possible fix is to avoid using anonymous classes, use normal classes instead.
A similar case:
[https://stackoverflow.com/questions/36902471/inner-class-reference-to-enclosing-class-instance-is-null]
Avro: 1.10.2
Kryo: 2.24.0
Flink: 1.12.4
was:
We have an Apache Flink application, that processes Avro GenericRecords. Flink
uses Kryo serialization. The serialization fails:
com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
Serialization trace:
props (org.apache.avro.Schema$Field)
fieldMap (org.apache.avro.Schema$RecordSchema)
schema (org.apache.avro.generic.GenericData$Record)
values (org.apache.avro.generic.GenericData$Record)
values (org.apache.avro.generic.GenericData$Record)
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:95)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
~[kryo-2.24.0.jar:?]
...
Caused by: java.lang.NullPointerException
at org.apache.avro.JsonProperties$2$1$1.<init>(JsonProperties.java:175)
~[avro-1.10.2.jar:1.10.2]
at org.apache.avro.JsonProperties$2$1.iterator(JsonProperties.java:174)
~[avro-1.10.2.jar:1.10.2]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:80)
~[kryo-2.24.0.jar:?]
at
com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
~[kryo-2.24.0.jar:?]
at com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
~[kryo-2.24.0.jar:?]
...
This problem occurs, because Kryo uses Unsafe to copy objects. When creating an
object, Unsafe does not call the class constructor. Therefore the field *props*
in the class JsonProperties is not properly initialized.
The field *props* is implemented via anonymous inner classes. The
NullPointerException occurs at JsonProperties.java:175, because the reference
to the enclosing instance of AbstractSet is null.
A possible fix is to avoid using anonymous classes, use normal classes instead.
A similar case:
[https://stackoverflow.com/questions/36902471/inner-class-reference-to-enclosing-class-instance-is-null]
Avro: 1.10.2
Kryo: 2.24.0
Flink: 1.12.4
> NPE when serializing Avro GenericRecord with Kryo
> -------------------------------------------------
>
> Key: AVRO-3438
> URL: https://issues.apache.org/jira/browse/AVRO-3438
> Project: Apache Avro
> Issue Type: Bug
> Affects Versions: 1.11.0, 1.10.2
> Reporter: Arseniy Tashoyan
> Priority: Major
>
> We have an Apache Flink application, that processes Avro GenericRecords.
> Flink uses Kryo serialization. The serialization fails:
>
> com.esotericsoftware.kryo.KryoException: java.lang.NullPointerException
> Serialization trace:
> props (org.apache.avro.Schema$Field)
> fieldMap (org.apache.avro.Schema$RecordSchema)
> schema (org.apache.avro.generic.GenericData$Record)
> values (org.apache.avro.generic.GenericData$Record)
> values (org.apache.avro.generic.GenericData$Record)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:82)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
> ~[kryo-2.24.0.jar:?]
> at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:599)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:95)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21)
> ~[kryo-2.24.0.jar:?]
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.write(FieldSerializer.java:495)
> ~[kryo-2.24.0.jar:?]
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
> ~[kryo-2.24.0.jar:?]
> ...
> Caused by: java.lang.NullPointerException
> at org.apache.avro.JsonProperties$2$1$1.<init>(JsonProperties.java:175)
> ~[avro-1.10.2.jar:1.10.2]
> at org.apache.avro.JsonProperties$2$1.iterator(JsonProperties.java:174)
> ~[avro-1.10.2.jar:1.10.2]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:80)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.MapSerializer.write(MapSerializer.java:21)
> ~[kryo-2.24.0.jar:?]
> at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:523)
> ~[kryo-2.24.0.jar:?]
> at
> com.esotericsoftware.kryo.serializers.ObjectField.write(ObjectField.java:61)
> ~[kryo-2.24.0.jar:?]
> ...
>
> This problem occurs, because Kryo uses Unsafe to copy objects. When creating
> an object, Unsafe does not call the class constructor. Therefore the field
> *props* in the class JsonProperties is not properly initialized. The field
> *props* is implemented via anonymous inner classes. The NullPointerException
> occurs at JsonProperties.java:175, because the reference to the enclosing
> instance of AbstractSet is null.
> A possible fix is to avoid using anonymous classes, use normal classes
> instead. A similar case:
> [https://stackoverflow.com/questions/36902471/inner-class-reference-to-enclosing-class-instance-is-null]
>
> Avro: 1.10.2
> Kryo: 2.24.0
> Flink: 1.12.4
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
