[
https://issues.apache.org/jira/browse/AVRO-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan Skraba updated AVRO-2799:
------------------------------
Summary: [Java] protobuf map types are not supported (was: Endless
recursive calls by using ProtobufDatumReader when decoding proto map)
> [Java] protobuf map types are not supported
> -------------------------------------------
>
> Key: AVRO-2799
> URL: https://issues.apache.org/jira/browse/AVRO-2799
> Project: Apache Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.9.2
> Reporter: Allen Wang
> Priority: Major
>
> I have a protobuf message defined as this:
>
> {code:java}
> syntax = "proto3";
> package ExampleProtobuf;
> option java_package = "com.example";
> import "google/protobuf/any.proto";
> import "google/protobuf/timestamp.proto";
> import "google/protobuf/wrappers.proto";
> enum EventType {
> TYPE_UNKNOWN = 0;
> TYPE_ORDER_RECEIVED = 1;
> TYPE_PAYMENT_AUTHORIZED = 2;
> TYPE_PAYMENT_CAPTURED = 3;
> TYPE_ORDER_SUBMITTED = 4;
> TYPE_ORDER_DELIVERY_CREATED = 5;
> TYPE_MERCHANT_CONFIRM_ORDER = 6;
> }
> message RevenueEvent {
> google.protobuf.StringValue id = 1;
> EventType type = 2;
> google.protobuf.Timestamp change_time = 3;
> google.protobuf.StringValue country = 4;
> map <string, google.protobuf.Any> payload = 5;
> google.protobuf.StringValue source = 6;
> bool is_test = 7;
> google.protobuf.StringValue version = 8;
> google.protobuf.StringValue checksum = 9;
> }
> {code}
>
>
> I wrote a program to encode such Protobuf message with Avro schema and
> convert it back to Protobuf message:
>
>
> {code:java}
> public class EventTest {
> public static void main(String[] args) throws Exception {
> Example.RevenueEvent event = RevenueEvent.newBuilder()
> .setId(StringValue.newBuilder()
> .setValue("12345")
> .build())
> .setChangeTime(Timestamp.newBuilder()
> .setSeconds(System.currentTimeMillis())
> .build())
> .setChecksum(StringValue.newBuilder()
> .setValue("xyz")
> .build())
> .setCountry(StringValue.newBuilder()
> .setValue("US")
> .build())
> .setSource(StringValue.newBuilder()
> .setValue("unknown")
> .build())
> .setType(EventType.TYPE_MERCHANT_CONFIRM_ORDER)
> .putPayload("key",
> Any.newBuilder().setValue(ByteString.copyFromUtf8("xyz")).build())
> .build();
> ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
> Encoder encoder = EncoderFactory.get().directBinaryEncoder(outputStream,
> null);
> GenericDatumWriter<Example.RevenueEvent> datumWriter = new
> ProtobufDatumWriter<Example.RevenueEvent>(RevenueEvent.class);
> datumWriter.write(event, encoder);
> encoder.flush();
> byte[] bytes = outputStream.toByteArray();
> DatumReader<RevenueEvent> datumReader = new
> ProtobufDatumReader<>(RevenueEvent.class);
> InputStream inputStream = new ByteArrayInputStream(bytes);
> Decoder decoder = DecoderFactory.get().binaryDecoder(inputStream, null);
> datumReader.read(null, decoder);
> }
> }
> {code}
>
>
> The program crashed with java.lang.StackOverflowError:
>
> Exception in thread "main" java.lang.StackOverflowErrorException in thread
> "main" java.lang.StackOverflowError at
> java.base/java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at org.apache.avro.specific.SpecificData.getClass(SpecificData.java:250) at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:141) at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143) at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143) at
> org.apache.avro.protobuf.ProtobufData.newRecord(ProtobufData.java:143)
> ...
>
> Looking into ProtoData.java, there seems to be a situation where the
> recursion would not end:
>
> {code:java}
> @Override
> public Object newRecord(Object old, Schema schema) {
> try {
> Class c = SpecificData.get().getClass(schema);
> if (c == null)
> return newRecord(old, schema); // punt to generic <-- endless recursion
> if (c.isInstance(old))
> return old; // reuse instance
> return c.getMethod("newBuilder").invoke(null);
> } catch (Exception e) {
> throw new RuntimeException(e);
> }
> }
> {code}
> Further experiment shows that if the map payload is not set, the decoding
> would be successful.
>
> protobuf-java:3.8.0 is used.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
