[
https://issues.apache.org/jira/browse/AVRO-1865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Kasper updated AVRO-1865:
-----------------------------------
Description:
When trying to serialize Java classes generated by Avro that contain a field as
follows:
{{array<MyClass> myclasses = []}}
serialization fails with the following error message
{{com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
no-arg constructor)}}
Minimum example to recreate this:
Avro IDL definition:
{code:java}
@namespace("com.adello")
protocol KryoTest {
record TestRecord {
array<int> values = [];
}
}
{code}
This gets compiled using {{java -jar avro-tools-1.8.1.jar idl2schemata
kryo.avdl .}}
Resulting Avro schema:
{code:javascript}
{
"type" : "record",
"name" : "TestRecord",
"namespace" : "com.adello",
"fields" : [ {
"name" : "values",
"type" : {
"type" : "array",
"items" : "int"
},
"default" : [ ]
} ]
}
{code}
Generate Java class using {{java -jar avro-tools-1.8.1.jar compile schema
TestRecord.avsc}}
Kryo serialization test code:
{code:java}
Kryo kryo = new Kryo();
try {
Output output = new Output(new FileOutputStream("test.bin"));
TestRecord o = TestRecord.newBuilder().build();
System.out.println("Before serialization: ");
System.out.println(o.toString());
kryo.writeObject(output, o);
output.close();
Input input = new Input(new FileInputStream("test.bin"));
TestRecord i = kryo.readObject(input, TestRecord.class);
input.close();
System.out.println("After deserialization: ");
System.out.println(i.toString());
System.out.println(Objects.deepEquals(o, i));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
{code}
This is a common enough issue with Kryo that they have a dedicated section on
their website on it:
http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
The attached patch adds a no args constructor for the inner class Array of
GenericData with zero entries and a dummy schema. My personal tests for
serialization with Kryo were successful when using it. Since I do not have
complete insight into Avro I'd like to know if this could be a breaking change
and how to test it if so.
was:
When trying to serialize Java classes generated by Avro that contain a field as
follows:
{{array<MyClass> myclasses = []}}
serialization fails with the following error message
{{com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
no-arg constructor)}}
Minimum example to recreate this:
Avro IDL definition:
{code:java}
@namespace("com.adello")
protocol KryoTest {
record TestRecord {
array<int> values = [];
}
}
{code}
This gets compiled using {{java -jar avro-tools-1.8.1.jar idl2schemata
kryo.avdl .}}
Resulting Avro schema:
{code:javascript}
{
"type" : "record",
"name" : "TestRecord",
"namespace" : "com.adello",
"fields" : [ {
"name" : "values",
"type" : {
"type" : "array",
"items" : "int"
},
"default" : [ ]
} ]
}
{code}
Kryo serialization test code:
{code:java}
Kryo kryo = new Kryo();
try {
Output output = new Output(new FileOutputStream("test.bin"));
TestRecord o = TestRecord.newBuilder().build();
System.out.println("Before serialization: ");
System.out.println(o.toString());
kryo.writeObject(output, o);
output.close();
Input input = new Input(new FileInputStream("test.bin"));
TestRecord i = kryo.readObject(input, TestRecord.class);
input.close();
System.out.println("After deserialization: ");
System.out.println(i.toString());
System.out.println(Objects.deepEquals(o, i));
} catch (FileNotFoundException e) {
e.printStackTrace();
}
{code}
This is a common enough issue with Kryo that they have a dedicated section on
their website on it:
http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
The attached patch adds a no args constructor for the inner class Array of
GenericData with zero entries and a dummy schema. My personal tests for
serialization with Kryo were successful when using it. Since I do not have
complete insight into Avro I'd like to know if this could be a breaking change
and how to test it if so.
> GenericData.Array class missing no arg constructor for Kryo serialization
> -------------------------------------------------------------------------
>
> Key: AVRO-1865
> URL: https://issues.apache.org/jira/browse/AVRO-1865
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.8.1
> Reporter: Alexander Kasper
> Labels: newbie, patch
> Attachments: add-no-arg-ctor-genericdata-array.diff
>
>
> When trying to serialize Java classes generated by Avro that contain a field
> as follows:
> {{array<MyClass> myclasses = []}}
> serialization fails with the following error message
> {{com.esotericsoftware.kryo.KryoException: Class cannot be created (missing
> no-arg constructor)}}
> Minimum example to recreate this:
> Avro IDL definition:
> {code:java}
> @namespace("com.adello")
> protocol KryoTest {
> record TestRecord {
> array<int> values = [];
> }
> }
> {code}
> This gets compiled using {{java -jar avro-tools-1.8.1.jar idl2schemata
> kryo.avdl .}}
> Resulting Avro schema:
> {code:javascript}
> {
> "type" : "record",
> "name" : "TestRecord",
> "namespace" : "com.adello",
> "fields" : [ {
> "name" : "values",
> "type" : {
> "type" : "array",
> "items" : "int"
> },
> "default" : [ ]
> } ]
> }
> {code}
> Generate Java class using {{java -jar avro-tools-1.8.1.jar compile schema
> TestRecord.avsc}}
> Kryo serialization test code:
> {code:java}
> Kryo kryo = new Kryo();
> try {
> Output output = new Output(new FileOutputStream("test.bin"));
> TestRecord o = TestRecord.newBuilder().build();
> System.out.println("Before serialization: ");
> System.out.println(o.toString());
> kryo.writeObject(output, o);
> output.close();
> Input input = new Input(new FileInputStream("test.bin"));
> TestRecord i = kryo.readObject(input, TestRecord.class);
> input.close();
> System.out.println("After deserialization: ");
> System.out.println(i.toString());
> System.out.println(Objects.deepEquals(o, i));
> } catch (FileNotFoundException e) {
> e.printStackTrace();
> }
> {code}
> This is a common enough issue with Kryo that they have a dedicated section on
> their website on it:
> http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception
> The attached patch adds a no args constructor for the inner class Array of
> GenericData with zero entries and a dummy schema. My personal tests for
> serialization with Kryo were successful when using it. Since I do not have
> complete insight into Avro I'd like to know if this could be a breaking
> change and how to test it if so.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
