[
https://issues.apache.org/jira/browse/IMPALA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16764551#comment-16764551
]
ASF subversion and git services commented on IMPALA-5031:
---------------------------------------------------------
Commit 8b8d935a0045aba5caea4106a42fdc647a2a6147 in impala's branch
refs/heads/master from Jim Apple
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=8b8d935 ]
IMPALA-5031: `uint8_t & int` type is int
UBSAN finds the following in ParquetBoolDecoder.TestDecodeAndSkipping:
util/bit-stream-utils.inline.h:156:25: runtime error: left shift of 42 by 28
places cannot be represented in type 'int'
#0 BatchedBitReader::GetUleb128Int(unsigned int*)
util/bit-stream-utils.inline.h:156:25
#1 RleBatchDecoder<bool>::NextCounts() util/rle-encoding.h:778:40
#2 RleBatchDecoder<bool>::NextNumRepeats() util/rle-encoding.h:622:28
#3 RleBatchDecoder<bool>::GetValues(int, bool*) util/rle-encoding.h:858:27
#4 bool ParquetBoolDecoder::DecodeValue<(parquet::Encoding::type)3>(bool*)
exec/parquet/parquet-bool-decoder.h:85:24
#5 TestSkipping(parquet::Encoding::type, unsigned char*, int, vector<bool>
const&, int, int)::$_0::operator()() const
exec/parquet/parquet-bool-decoder-test.cc:59
#6 TestSkipping(parquet::Encoding::type, unsigned char*, int, vector<bool>
const&, int, int) exec/parquet/parquet-bool-decoder-test.cc:69:221
#7 ParquetBoolDecoder_TestDecodeAndSkipping_Test::TestBody()
exec/parquet/parquet-bool-decoder-test.cc:85:5
#9 testing::Test::Run()
(/home/ubuntu/Impala/be/build/debug/exec/parquet/parquet-bool-decoder-test+0x6ee4f09)
The problem is the line
*v |= (byte & 0x7F) << shift;
byte is an uint8_t and 0x7F is an int. The standard section
[expr.bit.and] then applies the "usual arithmetic conversions"
specified in [expr], which applies "if the type of the operand with
signed integer type can represent all of the values of the type of the
operand with unsigned integer type, the operand with unsigned integer
type shall be converted to the type of the operand with signed integer
type." That makes byte & 0x7F a signed integer type, and [expr.shift]
says that "if E1 has a signed type and non-negative value, and E1×2^E2
is representable in the corresponding unsigned type of the result
type, then that value, converted to the result type, is the resulting
value; otherwise, the behavior is undefined."
Change-Id: Ie6e0b956751090f3f8aadd6783b5e06e55e57abe
Reviewed-on: http://gerrit.cloudera.org:8080/12346
Reviewed-by: Jim Apple <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> UBSAN clean and method for testing UBSAN cleanliness
> ----------------------------------------------------
>
> Key: IMPALA-5031
> URL: https://issues.apache.org/jira/browse/IMPALA-5031
> Project: IMPALA
> Issue Type: Task
> Components: Backend, Infrastructure
> Affects Versions: Impala 2.9.0
> Reporter: Jim Apple
> Assignee: Jim Apple
> Priority: Minor
>
> http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBehaviorSanitizer.html
> builds are supported after https://gerrit.cloudera.org/#/c/6186/, but
> Impala's test suite triggers many errors under UBSAN. Those errors should be
> fixed and then there should be a way to run the test suite under UBSAN and
> fail if there were any errors detected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
