[
https://issues.apache.org/jira/browse/FLINK-11333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755185#comment-16755185
]
Yun Tang commented on FLINK-11333:
----------------------------------
I planed to use avro-protobuf to extract Protobuf message's schema to verify
whether schema evolvable in avro's view. However, from [the
comment|https://github.com/confluentinc/schema-registry/pull/672#issuecomment-393348920]
described, avro defines different rules for compatibility. For example, Long
cannot be treated as compatibly with Integer for avro, while protobuf could.
And I write a simple program to verify this, there really exists difference.
I'm afraid the idea to use avro to judge whether two protobuf message
compatible is not correct. As far as I could see, check protobuf message
compatibility seems a non-trivial work, we might have to leave schema evolution
check of protobuf during job runtime.
> First-class support for Protobuf types with evolvable schema
> ------------------------------------------------------------
>
> Key: FLINK-11333
> URL: https://issues.apache.org/jira/browse/FLINK-11333
> Project: Flink
> Issue Type: Sub-task
> Components: Type Serialization System
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Yun Tang
> Priority: Major
>
> I think we have more and more users who are thinking about using Protobuf for
> their state types.
> Right now, Protobuf isn't supported directly in Flink. The only way to use
> Protobuf for a type is to register it via Kryo:
> https://ci.apache.org/projects/flink/flink-docs-release-1.7/dev/custom_serializers.html.
> Likewise for Avro types, we should be able to natively support Protobuf,
> having a {{ProtobufSerializer}} that handles serialization of Protobuf types.
> The serializer should also write necessary information in its snapshot, to
> enable schema evolution for it in the future. For Protobuf, this should
> almost work out-of-the-box.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
