[
https://issues.apache.org/jira/browse/AVRO-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17128517#comment-17128517
]
David Mollitor commented on AVRO-2780:
--------------------------------------
I would think so! Thanks for bringing to my attention.
I've seen something similar before. Avro attempts to load classes unexpectedly
depending on the use case. Here, the class loading was failing and then trying
to fall back. The fall back was where things were not working. Try out your
use case again and see if this resolves it. Otherwise there is another issue
at play here and the library should not be falling back.
> ProtobufData and ThriftData Can Get Into Endless Loop
> -----------------------------------------------------
>
> Key: AVRO-2780
> URL: https://issues.apache.org/jira/browse/AVRO-2780
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.9.2
> Reporter: Tianyu Lang
> Assignee: David Mollitor
> Priority: Critical
> Fix For: 1.10.0
>
>
> We have found a bug related, but probably not limited to ProtobufDatumReader.
> In Protobuf, if the ".proto" file has the same name as the message, in the
> generated ".java" file, "OuterClass" will be appended to the outer class's
> name. This is documented here:
> [https://developers.google.com/protocol-buffers/docs/reference/java-generated#invocation]
> Now we have a test protobuf class defined like so:
>
> {code:java}
> syntax = "proto2";
> package test.avrobug;
> option java_package = "com.test.protos.avrobug";
> option java_generic_services = true;
> enum State {
> BAD = 0;
> VERY_BAD = 1;
> SUPER_BAD = 2;
> }
> message Dummy {
> optional string token = 1;
> optional string description = 2;
> optional int64 count = 3;
> optional State state = 4;
> }
> {code}
>
>
> If we first create a Protobuf object, write it to a file as Avro through
> ProtobufDatumReader, then read it into a Protobuf with ProtobufDatumReader, a
> stack overflow exception will happen. Code to reproduce is as follows:
>
> {code:java}
> @Test
> public void ProtoToAvroOuterClassBug() throws Exception {
> DummyOuterClass.Dummy dummy = DummyOuterClass.Dummy.newBuilder()
> .setCount(50)
> .setDescription("hehe")
> .setToken("abc123")
> .setState(DummyOuterClass.State.BAD)
> .build();
>
> ProtobufDatumWriter<DummyOuterClass.Dummy> pbWriter =
> new ProtobufDatumWriter<>(DummyOuterClass.Dummy.class);
> DataFileWriter<DummyOuterClass.Dummy> dataFileWriter = new
> DataFileWriter<>(pbWriter);
> Schema schema = ProtobufData.get().getSchema(DummyOuterClass.Dummy.class);
> dataFileWriter.create(schema,
> new File("/Users/me/Documents/DummyAvroNoDefaultValues.avro"));
> dataFileWriter.append(dummy);
> dataFileWriter.close();
> ProtobufDatumReader<DummyOuterClass.Dummy> pbReader =
> new ProtobufDatumReader<>(DummyOuterClass.Dummy.class);
> DataFileReader<DummyOuterClass.Dummy> dataFileReader =
> new DataFileReader<>(new
> File("/Users/me/Documents/DummyAvroNoDefaultValues.avro"),
> pbReader);
> while(dataFileReader.hasNext()) {
> DummyOuterClass.Dummy record = dataFileReader.next();
> String recordStr = record.toString();
> String originStr = dummy.toString();
> System.out.println(recordStr);
> System.out.println(originStr);
> }
> }
> {code}
>
>
> When this is run, a stack overflow exception will happen with the following
> stack trace:
>
> {code:java}
> java.lang.StackOverflowErrorjava.lang.StackOverflowError
> at
> java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at
> org.apache.avro.specific.SpecificData.getClass(SpecificData.java:250)
> at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:141)
> at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143)
> at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143)
> at
> ..................{code}
> newRecord() is recursed infinitely until stack explodes
> I did a bit code tracing and I found out when Avro tries to load the
> corresponding Java class from the schema, it tries to load the class
> "com.test.protos.avrobug.Dummy.Dummy". I suspect the double "Dummy" at the
> end is related to the stack overflow, but I don't have enough knowledge on
> Proto or Avro to know the exact reason.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
