[
https://issues.apache.org/jira/browse/IMPALA-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824876#comment-17824876
]
ASF subversion and git services commented on IMPALA-12845:
----------------------------------------------------------
Commit b18999fe0975a0795c4b10243d7c5465ebef9ed3 in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b18999fe0 ]
IMPALA-12845: Crash with DESCRIBE on a complex type from an Iceberg table
A DESCRIBE statement on a complex column contained in an Iceberg table
runs into a DCHECK and crashes Impala. An example with an array:
describe functional_parquet.iceberg_resolution_test_external.phone
Note that this also happens with Iceberg metadata tables, for example:
describe functional_parquet.iceberg_query_metadata.\
entries.readable_metrics;
With non-Iceberg tables there is no error.
The problem is that for Iceberg tables, the DESCRIBE statement returns
four columns: "name", "type", "comment" and "nullable" (only Iceberg and
Kudu tables have "nullable"). However, the DESCRIBE statement response
for complex types only contains the first three columns, i.e. no column
for "nullable". But as the table is an Iceberg table, the 'metadata_'
field of HS2ColumnarResultSet is still populated with four columns.
The DCHECK in HS2ColumnarResultSet::AddOneRow() expects the number of
columns to be the same in the DESCRIBE statement response and the
'metadata_' field.
This commit solves the problem by only adding the "nullable" column to
the 'metadata_' field if the target of the DESCRIBE statement is a
table, not a complex type.
Note that Kudu tables do not support complex types so this issue does
not arise there.
This change also addresses a minor issue: DescribeTableStmt::analyze()
did not check whether the statement was already analyzed and did not set
the 'analyzer_' field which would indicate that analysis had already
been done. This is now corrected.
Testing:
- added tests in describe-path.test for arrays, maps and structs from
regular Iceberg tables and metadata tables.
Change-Id: I5eda21a41167cc1fda183aa16fd6276a6a16f5d3
Reviewed-on: http://gerrit.cloudera.org:8080/21105
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Crash with DESCRIBE on a complex type from an Iceberg table
> -----------------------------------------------------------
>
> Key: IMPALA-12845
> URL: https://issues.apache.org/jira/browse/IMPALA-12845
> Project: IMPALA
> Issue Type: Bug
> Reporter: Daniel Becker
> Assignee: Daniel Becker
> Priority: Critical
>
> A DESCRIBE statement on a struct contained in an Iceberg table crashes Impala:
> In Hive:
> {code:java}
> create table ice_struct (id INT, s STRUCT<i: INT>) stored by iceberg;{code}
> In Impala:
> {code:java}
> describe ice_struct.s
> Caught exception TSocket read 0 bytes, type=<class
> 'thrift.transport.TTransport.TTransportException'> in FetchResults.
> Error communicating with impalad: TSocket read 0 bytes{code}
> The logs in impalad.FATAL indicate that we hit a DCHECK:
> {code:java}
> F0227 11:53:49.936864 680819 query-result-set.cc:386] Check failed: num_col
> == metadata_.columns.size() (3 vs. 4){code}
>
> Note that this also happens with Iceberg metadata tables, for example:
> {code:java}
> describe
> functional_parquet.iceberg_query_metadata.entries.readable_metrics;{code}
> With non-Iceberg tables there is no error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
