Quanlong Huang created IMPALA-13865:
---------------------------------------
Summary: Frontend operations should handle invalidated or failed
loaded tables
Key: IMPALA-13865
URL: https://issues.apache.org/jira/browse/IMPALA-13865
Project: IMPALA
Issue Type: Bug
Components: Frontend
Reporter: Quanlong Huang
Assignee: Quanlong Huang
For Frontend statements like SHOW TABLE STATS, SHOW FILES, LOAD DATA, etc.,
it's possible that when analyzing the statement and generating the
TExecRequest, the table is loaded and good. But when executing the TExecRequest
in Frontend operations like doGetTableStats(), doGetTableFiles(),
doLoadTableData(), etc., the table becomes invalidated or failed loaded due to
other concurrent requests. Such cases are not handled and we will hit errors
discribed below.
h4. SHOW TABLE STATS / SHOW PARTITIONS
in local catalog mode:
{code:java}
InternalException: Invalid table class: class
org.apache.impala.catalog.local.FailedLoadLocalTable{code}
in legacy catalog mode:
{code:java}
InternalException: Invalid table class: class
org.apache.impala.catalog.IncompleteTable {code}
h4. SHOW FILES
{code:java}
InternalException: SHOW FILES only supports Hdfs table. Unsupported table
class: class org.apache.impala.catalog.local.FailedLoadLocalTable{code}
in legacy catalog mode:
{code:java}
InternalException: SHOW FILES only supports Hdfs table. Unsupported table
class: class org.apache.impala.catalog.IncompleteTable {code}
h4. SHOW COLUMN STATS
{code:java}
InternalException: Required field 'rows' was not present! Struct:
TResultSet(rows:null,
schema:TResultSetMetadata(columns:[TColumn(columnName:Column,
columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:STRING))]), virtual_column_type:NONE),
TColumn(columnName:Type, columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:STRING))]), virtual_column_type:NONE),
TColumn(columnName:#Distinct Values,
columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))]), virtual_column_type:NONE),
TColumn(columnName:#Nulls, columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))]), virtual_column_type:NONE),
TColumn(columnName:Max Size,
columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))]), virtual_column_type:NONE),
TColumn(columnName:Avg Size,
columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:DOUBLE))]), virtual_column_type:NONE),
TColumn(columnName:#Trues, columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))]), virtual_column_type:NONE),
TColumn(columnName:#Falses,
columnType:TColumnType(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))]), virtual_column_type:NONE)])){code}
h4. DESCRIBE
no errors but with empty result:
{code:java}
Fetched 0 row(s){code}
h4. LOAD DATA into a non-partitioned table
{code:java}
NullPointerException: null{code}
thrown here since the msTbl is null:
{code:java}
destPathString = catalog.getTable(tableName.getDb(), tableName.getTbl())
.getMetaStoreTable().getSd().getLocation();{code}
[https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/fe/src/main/java/org/apache/impala/service/Frontend.java#L1006]
h4. LOAD DATA into a partition
{code:java}
PartitionNotFoundException: Partition not found: TPartitionKeyValue(name:year,
value:2009), TPartitionKeyValue(name:month, value:1){code}
h4. LOAD DATA into an Iceberg table
Coordinator hits a DCHECK in the DEBUG build:
{code:java}
F0315 12:50:57.754804 89419 child-query.h:60]
2541f70af3c295bc:2160088800000000] Check failed: !query_.empty(){code}
[https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/be/src/service/child-query.h#L60]
In RELEASE build, just a NullPointerException
{code:java}
NullPointerException: null{code}
thrown from
{code:java}
String destPathString = catalog.getTable(tableName.getDb(),
tableName.getTbl())
.getMetaStoreTable().getSd().getLocation();{code}
[https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/fe/src/main/java/org/apache/impala/service/Frontend.java#L1057]
h4. SHOW METADATA TABLES IN an Iceberg table
{code:java}
IllegalStateException: null{code}
hits this:
{code:java}
// Analysis ensures that only Iceberg tables are passed to this function.
Preconditions.checkState(catalogTbl instanceof FeIcebergTable);{code}
[https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/fe/src/main/java/org/apache/impala/service/Frontend.java#L1333]
h4. DESCRIBE HISTORY on an Iceberg table
{code:java}
ClassCastException: org.apache.impala.catalog.local.FailedLoadLocalTable cannot
be cast to org.apache.impala.catalog.FeIcebergTable{code}
thrown here:
{code:java}
FeTable feTable = getCatalog().getTable(params.getTable_name().db_name,
params.getTable_name().table_name);
FeIcebergTable feIcebergTable = (FeIcebergTable) feTable; {code}
[https://github.com/apache/impala/blob/356b7e5ddf7868968fb76ca55a8046d0291388fd/fe/src/main/java/org/apache/impala/service/Frontend.java#L1493]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
