I am doing a Java prototype where the publisher converts Protobuf objects
into Avro records to send to a stream, then the consumer deserializes them
back into Protobuf objects. The Protobuf schemas on the publisher and
consumer are the "source of truth" schemas, and I  use "Schema schema =
ProtobufData.get().getSchema(MyProtobufClass.class);" to get the
corresponding Avro schema to convert back and forth.
However, one problem I encountered is: the Avro schema resulted from this
method does not have a default value for repeated (array) Protobuf fields.
This is causing compatibility issues for me: if I update the consumer
schema first by adding a new repeated Protobuf field, the generated Avro
schema cannot handle old messages without this field, as it does not have a
default value to fill in for the new array field.
I can probably work around this by traversing the Avro Schema object and
add a default value for all the array fields, but this feels a bit clumsy.

Is there a better way to do this?
Also, what's the reason behind having no default values for array field?
(Am I going into compatibility edge cases by forcing default values on
array fields?)

Thank you

Reply via email to