[
https://issues.apache.org/jira/browse/IMPALA-13892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18009314#comment-18009314
]
ASF subversion and git services commented on IMPALA-13892:
----------------------------------------------------------
Commit 07a5716773d410b84be4f97ef5bede2f65b81546 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=07a571677 ]
IMPALA-13892: Add support for printing STRUCTs
Tuple cache correctness verification is failing as the code in
debug-util.cc used for printing the text version of tuples does
not support printing structs. It hits a DCHECK and kills Impala.
This adds supports for printing structs to debug-util.cc, fixing
tuple cache correctness verification for complex types. To print
structs correctly, each slot needs to know its field name. The
ColumnType has this information, but it requires a field idx to
lookup the name. This is the last index in the absolute path for
this slot. However, the materialized path can be truncated to
remove some indices at the end. Since we need that information to
resolve the field name, this adds the struct field idx to the
TSlotDescriptor to pass it to the backend.
This also adds a counter to the profile to track when correctness
verification is on. This is useful for testing.
Testing:
- Added a custom cluster test using nested types with
correctness verification
- Examined some of the text files
Change-Id: Ib9479754c2766a9dd6483ba065e26a4d3a22e7e9
Reviewed-on: http://gerrit.cloudera.org:8080/23075
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Daniel Becker <[email protected]>
Tested-by: Joe McDonnell <[email protected]>
> Tuple caching correctness verification can't handle STRUCT types
> ----------------------------------------------------------------
>
> Key: IMPALA-13892
> URL: https://issues.apache.org/jira/browse/IMPALA-13892
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
>
> Tuple caching correctness verification relies on debug-util's PrintRow().
> PrintRow() can write some collection types, but it does not currently write
> structs. With a fix for IMPALA-13886, running the following SQL twice crashes
> Impala:
> {noformat}
> select nested_struct from functional_parquet.complextypestbl;{noformat}
> It hits this DCHECK:
> {noformat}
> F0324 18:13:52.138927 1981618 raw-value.cc:517]
> c440c38c551d001d:ee76eec100000001] Check failed: false Unknown type:
> STRUCT{noformat}
> The crash stack is:
> {noformat}
> 6 impalad!google::LogMessageFatal::~LogMessageFatal() [logging.cc : 2048 +
> 0x5]
> 7 impalad!impala::RawValue::PrintValue(void const*, impala::ColumnType
> const&, int, std::__cxx11::basic_stringstream<char, std::char_traits<char>,
> std::allocator<char> >*, bool) [raw-value.cc : 517 + 0x8]
> 8 impalad!impala::RawValue::PrintValue(void const*, impala::ColumnType
> const&, int, std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> >*) [raw-value.cc : 135 + 0x1b]
> 9 impalad!impala::PrintTuple[abi:cxx11](impala::Tuple const*,
> impala::TupleDescriptor const&) [debug-util.cc : 184 + 0x11]
> 10 impalad!impala::PrintRow[abi:cxx11](impala::TupleRow*,
> impala::RowDescriptor const&) [debug-util.cc : 198 + 0xf]
> 11 impalad!impala::TupleTextFileWriter::Write(impala::RowBatch*)
> [tuple-text-file-writer.cc : 50 + 0xc]
> 12 impalad!impala::TupleCacheNode::GetNext(impala::RuntimeState*,
> impala::RowBatch*, bool*) [tuple-cache-node.cc : 315 + 0xf]
> 13 impalad!impala::FragmentInstanceState::ExecInternal()
> [fragment-instance-state.cc : 446 + 0x12]
> 14 impalad!impala::FragmentInstanceState::Exec() [fragment-instance-state.cc
> : 104 + 0xc]
> 15 impalad!impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> [query-state.cc : 1013 + 0xc]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
