[
https://issues.apache.org/jira/browse/IMPALA-11502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17582074#comment-17582074
]
Andrew Sherman commented on IMPALA-11502:
-----------------------------------------
In IcebergCatalogOpExecutor:
{code}
public static void dropTable(FeIcebergTable feTable, boolean ifExists)
throws TableNotFoundException, ImpalaRuntimeException {
Preconditions.checkState(
IcebergTable.isSynchronizedTable(feTable.getMetaStoreTable()));
IcebergCatalog iceCatalog = IcebergUtil.getIcebergCatalog(feTable);
if (!iceCatalog.dropTable(feTable,
IcebergTable.isSynchronizedTable(feTable.getMetaStoreTable()))) {
// The table didn't exist.
if (!ifExists) {
throw new TableNotFoundException(String.format(
"Table '%s' does not exist in Iceberg catalog.",
feTable.getFullName()));
}
}
}
{code}
iceCatalog.dropTable returns false and so an exception is thrown.
That means that this code
https://github.com/apache/impala/blob/51126259bea2491316decef63d584f713d02a9a6/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L2707
throws an exception and we don't go on to drop the table from HMS.
I'm experimenting with having IcebergCatalogOpExecutor.dropTable() log a
warning instead of throwing. In these cases were we do (1) drop iceberg data
(2) drop hms reference then we have to be careful that (2) completes even if
(1) fails.
> Dropping files of Iceberg table in HadoopCatalog will cause DROP TABLE to fail
> ------------------------------------------------------------------------------
>
> Key: IMPALA-11502
> URL: https://issues.apache.org/jira/browse/IMPALA-11502
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 4.1.0
> Reporter: Gabor Kaszab
> Priority: Major
> Labels: iceberg, impala-iceberg
>
> When the files of n Iceberg table are dropped then a DROP TABLE will result
> in an error while the table will still show up in SHOW TABLES
> Here are the steps to repro:
> 1) Run from Impala-shell
> {code:java}
> DROP DATABASE IF EXISTS `drop_incomplete_table` CASCADE;
> CREATE DATABASE `drop_incomplete_table`;
> CREATE TABLE drop_incomplete_table.iceberg_tbl (i int) stored as iceberg
> tblproperties('iceberg.catalog'='hadoop.catalog',
>
> 'iceberg.catalog_location'='/test-warehouse/drop_incomplete_table');
> INSERT INTO drop_incomplete_table.iceberg_tbl VALUES (1), (2), (3);
> {code}
> 2) Drop the folder of the table with hdfs dfs
> {code:java}
> hdfs dfs -rm -r hdfs://localhost:20500/test-warehouse/drop_incomplete_table
> {code}
> 3) Try to drop the table from Impala-shell
> {code:java}
> DROP TABLE drop_incomplete_table.iceberg_tbl;
> {code}
> This results in the following error:
> {code:java}
> ERROR: TableNotFoundException: Table 'drop_incomplete_table.iceberg_tbl' does
> not exist in Iceberg catalog.
> {code}
> While table is still there in show tables output even after an invalidate
> metadata.
> Note, it's important for the repro to execute some SQL against the newly
> created table to load it in Impala. In this case I used an INSERT INTO but
> e.g. an ALTER TABLE would also be good. Apparently, when the table is
> "incomplete" (this is the state right after running CREATE TABLE) this works
> fine but not if the table is loaded.
> The suspicious part of code is in StmtMetadataLoader.loadTables() and
> getMissingTables() where there is a distinction between loaded and Incomplete
> tables.
> https://github.com/apache/impala/blob/2f74e956aa10db5af6a7cdc47e2ad42f63d5030f/fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java#L196
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
