Tim Armstrong created IMPALA-6946:
-------------------------------------
Summary: Hit DCHECK in impala::RleBatchDecoder<unsigned
int>::GetRepeatedValue
Key: IMPALA-6946
URL: https://issues.apache.org/jira/browse/IMPALA-6946
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 2.11.0, Impala 3.0, Impala 2.12.0, Impala 2.13.0,
Impala 3.1.0
Reporter: Tim Armstrong
Assignee: Tim Armstrong
The bug comes from conversion between signed and unsigned.
{noformat}
DCHECK_GT(num_repeats_to_consume, 0);
(gdb) p num_repeats_to_consume
$2 = -1003251240
{noformat}
{noformat}
#2 0x0000000001f422f0 in impala::ScalarColumnReader<impala::StringValue,
(parquet::Type::type)6, true>::ReadNonRepeatedValueBatch (this=0x6f547d50,
pool=0x7fb37bdbb330, max_values=0,
tuple_size=90, tuple_mem=0x0, num_values=0x7fb37bdbb330) at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:252
#3 0x0000000003fceea7 in google::LogMessage::Flush() ()
#4 0x0000000003fd246e in google::LogMessageFatal::~LogMessageFatal() ()
#5 0x0000000001f97197 in impala::RleBatchDecoder<unsigned
int>::GetRepeatedValue (this=0x1803fc6a0, num_repeats_to_consume=-1003251240)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/rle-encoding.h:493
#6 0x0000000001f96d7b in
impala::DictDecoder<impala::StringValue>::DecodeNextValue (this=0x1803fc698,
value=0x16b7e9000)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/dict-encoding.h:455
#7 0x0000000001f94528 in
impala::DictDecoder<impala::StringValue>::GetNextValue (value=0x16b7e9000,
this=0x1803fc698)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/dict-encoding.h:446
#8 impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6,
true>::ReadSlot<true, false> (pool=0x6f547d98, tuple=0x16b7e9000,
this=0x1803fc400)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:499
#9 impala::ScalarColumnReader<impala::StringValue, (parquet::Type::type)6,
true>::MaterializeValueBatch<false, true, false> (this=0x1803fc400,
pool=0x6f547d98, max_values=1, tuple_size=90,
tuple_mem=0x16b7e9000 "", num_values=0x7fb37bdbb5f0) at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:413
#10 0x0000000001f67b4a in impala::ScalarColumnReader<impala::StringValue,
(parquet::Type::type)6, true>::MaterializeValueBatch<false, true>
(this=0x1803fc400, pool=0x6f547d98, max_values=1,
tuple_size=90, tuple_mem=0x16b7e9000 "", num_values=0x7fb37bdbb5f0) at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:436
#11 0x0000000001f657c2 in impala::ScalarColumnReader<impala::StringValue,
(parquet::Type::type)6, true>::ReadValueBatch<false> (this=0x1803fc400,
pool=0x6f547d98, max_values=1, tuple_size=90,
tuple_mem=0x16b7e9000 "", num_values=0x6f547d50) at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:354
#12 0x0000000001f422f0 in impala::ScalarColumnReader<impala::StringValue,
(parquet::Type::type)6, true>::ReadNonRepeatedValueBatch (this=0x1803fc400,
pool=0x6f547d98, max_values=1,
tuple_size=90, tuple_mem=0x16b7e9000 "", num_values=0x6f547d50) at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet-column-readers.cc:252
#13 0x0000000001d70a1c in impala::HdfsParquetScanner::AssembleRows
(this=0x1a926000, column_readers=..., row_batch=0xdfc0780,
skip_row_group=0x1a9261b8)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:936
#14 0x0000000001d6d4f7 in impala::HdfsParquetScanner::GetNextInternal
(this=0x1a926000, row_batch=0xdfc0780)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:434
#15 0x0000000001d6b6c6 in impala::HdfsParquetScanner::ProcessSplit
(this=0x1a926000) at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-parquet-scanner.cc:332
#16 0x0000000001cf30e4 in impala::HdfsScanNode::ProcessSplit (this=0x10c21000,
filter_ctxs=..., expr_results_pool=0x7fb37bdbc480, scan_range=0xf3276c0,
scanner_thread_reservation=0x7fb37bdbc400)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:482
#17 0x0000000001cf2476 in impala::HdfsScanNode::ScannerThread (this=0x10c21000,
scanner_thread_reservation=90112)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:385
#18 0x0000000001cf18a8 in impala::HdfsScanNode::<lambda()>::operator()(void)
const (__closure=0x7fb37bdbcbc8)
at
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:300
{noformat}
I don't believe this will cause a crash on a release build and can only be
triggered by a very specific corrupt parquet file.
It was introduced by IMPALA-4177
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
