Jozef Vilcek created BEAM-4758:
----------------------------------
Summary: Avro-Protobuf support
Key: BEAM-4758
URL: https://issues.apache.org/jira/browse/BEAM-4758
Project: Beam
Issue Type: Wish
Components: io-java-avro
Reporter: Jozef Vilcek
Assignee: Eugene Kirpichov
I came a cross avro-protobuf support which allows to write Protobuf messages to
avro file and then read them back as a) GenericRecords or b) as Protobuf typed
message
Quite nice feature I am looking for. I do not know much about what is happening
behind the scenes, if there are not some hidden problems and limitations e.g.
on schema compatibility when evolving proto messages etc.
So what I want to ask is:
# Was this feature ever considered, to enable existing AvroIO code to use also
ProtobufDatumReader and ProtobufDatumWriter? Any wisdom what I should be aware
of?
# Would it make sense to add it to AvroIO ?
P.S.: I wanted to add it on the side for my needs and testing it but I did not
find an easy way of doing it. Writer infrastructure is easy to extend by
implementing custom FileIO.Sink but read side seems to be hard nut to crack
without copy-pasting much code from AvroIO and AvroSource.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
