junwen yang created HIVE-24454:
----------------------------------
Summary: Protobuf Mesage Incompatibility Detector
Key: HIVE-24454
URL: https://issues.apache.org/jira/browse/HIVE-24454
Project: Hive
Issue Type: Bug
Reporter: junwen yang
Attachments: hive_proto_incompatibility.txt
We have created a static checker which keeps track of the proto file change,
and detect potential incompatibility:
# Add/delete required field, which is the case reported in HBASE-25238.
# The tag number of a field has been changed, as described in HDFS-9788. Also,
the [protobuf guidelines
|https://developers.google.com/protocol-buffers/docs/proto]suggests _each field
in the message definition has a *unique number*. These numbers are used to
identify your fields in the [message binary
format|https://developers.google.com/protocol-buffers/docs/encoding], and
should not be changed once your message type is in use_.
# A required field has been changed to optional, or an optional field has
been changed to required. According to the guidelines in [protobuf official
website|https://developers.google.com/protocol-buffers/docs/proto], _*Required
Is Forever* You should be very careful about marking fields as {{required}}. If
at some point you wish to stop writing or sending a required field, it will be
problematic to change the field to an optional field - old readers will
consider messages without this field to be incomplete and may reject or drop
them unintentionally. You should consider writing application-specific custom
validation routines for your buffers instead._
We have applied our checker on the frequently maintained HIVE versions:
rel/release-1.2.2, rel/release-2.1.0, rel/release-2.1.1, rel/release-2.2.0,
rel/release-2.3.0, rel/release-2.3.1, rel/release-2.3.2, rel/release-2.3.3,
rel/release-2.3.4, rel/release-2.3.5, rel/release-2.3.6, rel/release-2.3.7,
rel/release-3.0.0, rel/release-3.1.0, rel/release-3.1.1, rel/release-3.1.2, and
we found 260 potential problems as attached.
The results reported by our checker got confirmation from developers in HBASE,
which can be found here HBASE-25340.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
