[
https://issues.apache.org/jira/browse/IMPALA-10370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
junwen yang updated IMPALA-10370:
---------------------------------
Description:
Regarding the issue IMPALA-8243 caused by the incompatibility of thrift
message, we have created a static checker which keeps track of the thrift file
change, and detect potential incompatibility:
# Add/delete required field. The thrift guidelines suggests _Any new fields
that you add should be optional_.
# The tag number of a field has been changed. Also, the thrift guidelines
suggests _Don’t change the numeric tags for any existing fields_.
# A required field has been changed to optional, or an optional field has
been changed to required. According to the guidelines , _*Required Is Forever*
You should be very careful about marking fields as required. If at some point
you wish to stop writing or sending a required field, it will be problematic to
change the field to an optional field — old readers will consider messages
without this field to be incomplete and may reject or drop them
unintentionally. You should consider writing application-specific custom
validation routines for your buffers instead. Some have come to the conclusion
that using required does more harm than good; they prefer to use only optional.
However, this view is not universal._
We have applied our checker on the frequently maintained IMPALA versions:
refs/tags/2.10.0, refs/tags/2.11.0, refs/tags/2.12.0, refs/tags/2.7.0,
refs/tags/2.8.0, refs/tags/2.9.0, refs/tags/3.0.0, refs/tags/3.0.1,
refs/tags/3.1.0, refs/tags/3.2.0, refs/tags/3.3.0, refs/tags/3.4.0, we found
more than 1000 problems as attached.
The results reported by our checker got confirmed by developers of HBASE and
our checker is requested by them, which can be found at HBASE-25340.
was:
Regarding the issue
[IMPALA-8243|https://issues.apache.org/jira/browse/IMPALA-8234] caused by the
incompatibility of thrift message, we have created a static checker which keeps
track of the thrift file change, and detect potential incompatibility:
# Add/delete required field. The thrift guidelines suggests _Any new fields
that you add should be optional_.
# The tag number of a field has been changed. Also, the thrift guidelines
suggests _Don’t change the numeric tags for any existing fields_.
# A required field has been changed to optional, or an optional field has
been changed to required. According to the guidelines , _*Required Is Forever*
You should be very careful about marking fields as required. If at some point
you wish to stop writing or sending a required field, it will be problematic to
change the field to an optional field — old readers will consider messages
without this field to be incomplete and may reject or drop them
unintentionally. You should consider writing application-specific custom
validation routines for your buffers instead. Some have come to the conclusion
that using required does more harm than good; they prefer to use only optional.
However, this view is not universal._
We have applied our checker on the frequently maintained IMPALA versions:
refs/tags/2.10.0, refs/tags/2.11.0, refs/tags/2.12.0, refs/tags/2.7.0,
refs/tags/2.8.0, refs/tags/2.9.0, refs/tags/3.0.0, refs/tags/3.0.1,
refs/tags/3.1.0, refs/tags/3.2.0, refs/tags/3.3.0, refs/tags/3.4.0, we found
more than 1000 problems as attached.
The results reported by our checker got confirmation from developers in HBASE,
which can be found here HBASE-25340.
> Thrift Message Incompatibility Detector
> ---------------------------------------
>
> Key: IMPALA-10370
> URL: https://issues.apache.org/jira/browse/IMPALA-10370
> Project: IMPALA
> Issue Type: Improvement
> Reporter: junwen yang
> Priority: Major
> Attachments: impala_thrift_incompatibility.txt
>
>
> Regarding the issue IMPALA-8243 caused by the incompatibility of thrift
> message, we have created a static checker which keeps track of the thrift
> file change, and detect potential incompatibility:
> # Add/delete required field. The thrift guidelines suggests _Any new fields
> that you add should be optional_.
> # The tag number of a field has been changed. Also, the thrift guidelines
> suggests _Don’t change the numeric tags for any existing fields_.
> # A required field has been changed to optional, or an optional field has
> been changed to required. According to the guidelines , _*Required Is
> Forever* You should be very careful about marking fields as required. If at
> some point you wish to stop writing or sending a required field, it will be
> problematic to change the field to an optional field — old readers will
> consider messages without this field to be incomplete and may reject or drop
> them unintentionally. You should consider writing application-specific custom
> validation routines for your buffers instead. Some have come to the
> conclusion that using required does more harm than good; they prefer to use
> only optional. However, this view is not universal._
> We have applied our checker on the frequently maintained IMPALA versions:
> refs/tags/2.10.0, refs/tags/2.11.0, refs/tags/2.12.0, refs/tags/2.7.0,
> refs/tags/2.8.0, refs/tags/2.9.0, refs/tags/3.0.0, refs/tags/3.0.1,
> refs/tags/3.1.0, refs/tags/3.2.0, refs/tags/3.3.0, refs/tags/3.4.0, we found
> more than 1000 problems as attached.
> The results reported by our checker got confirmed by developers of HBASE and
> our checker is requested by them, which can be found at HBASE-25340.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
