[
https://issues.apache.org/jira/browse/IMPALA-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16889840#comment-16889840
]
ASF subversion and git services commented on IMPALA-5031:
---------------------------------------------------------
Commit adaf344d48920c8a66a317c3720ca090f470fe5a in impala's branch
refs/heads/master from Jim Apple
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=adaf344 ]
IMPALA-5031: out-of-range enum values are undefined behavior
This patch fixes an out-of-range enum value in the end-to-end tests.
The [expr] section of the C++14 standard indicates that out-of-range
enum values are undefined behavior: "If during the evaluation of an
expression, the result is not mathematically defined or not in the
range of representable values for its type, the behavior is
undefined."
The [decl.enum] section explains what values are "in the range of
representable values for its type": "[F]or an enumeration where emin
is the smallest enumerator and emax is the largest, the values of the
enumeration are the values in the range bmin to bmax, defined as
follows: Let K be 1 for a two's complement representation and 0 for a
one's complement or sign-magnitude representation. bmax is the
smallest value greater than or equal to max(|emin| - K, |emax|) and
equal to 2M-1, where M is a non-negative integer. bmin is zero if emin
is non-negative and -(bmax+K) otherwise."
The Parquet PageType enum has emin = 0 and emax = 3, so bmin = 0 and
bmax = 3. The out-of-range value in the tests is 4294967249, and is
therefore undefined behavior. The interesting part of the backtrace
is:
parquet/parquet-column-readers.cc:1269:24: runtime error: load of
value 4294967249, which is not a valid value for type
'PageType::type'
#0 BaseScalarColumnReader::InitDictionary()
parquet/parquet-column-readers.cc:1269:24
#1 BaseScalarColumnReader::InitDictionaries(
vector<BaseScalarColumnReader*>)
parquet/parquet-column-readers.cc:1381:53
#2 HdfsParquetScanner::NextRowGroup()
parquet/hdfs-parquet-scanner.cc:678:14
#3 HdfsParquetScanner::GetNextInternal(RowBatch*)
parquet/hdfs-parquet-scanner.cc:437:45
#4 HdfsParquetScanner::ProcessSplit()
parquet/hdfs-parquet-scanner.cc:353:21
#5 HdfsScanNode::ProcessSplit(vector<FilterContext> const&,
MemPool*, io::ScanRange*, long*) exec/hdfs-scan-node.cc:514:21
#6 HdfsScanNode::ScannerThread(bool, long)
hdfs-scan-node.cc:415:7
#7 HdfsScanNode::ThreadTokenAvailableCb(ThreadResourcePool*)
::$_0::operator()() const hdfs-scan-node.cc:337:13
Change-Id: I2d126a8f3e5910d23088a3f916c4cf31aac28d95
Reviewed-on: http://gerrit.cloudera.org:8080/13805
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> UBSAN clean and method for testing UBSAN cleanliness
> ----------------------------------------------------
>
> Key: IMPALA-5031
> URL: https://issues.apache.org/jira/browse/IMPALA-5031
> Project: IMPALA
> Issue Type: Task
> Components: Backend, Infrastructure
> Affects Versions: Impala 2.9.0
> Reporter: Jim Apple
> Assignee: Jim Apple
> Priority: Minor
>
> http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBehaviorSanitizer.html
> builds are supported after https://gerrit.cloudera.org/#/c/6186/, but
> Impala's test suite triggers many errors under UBSAN. Those errors should be
> fixed and then there should be a way to run the test suite under UBSAN and
> fail if there were any errors detected.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
