Quanlong Huang created IMPALA-13983:
---------------------------------------
Summary: Invalidate+AlterTable could overwrite the result of a
concurrent AlterTable
Key: IMPALA-13983
URL: https://issues.apache.org/jira/browse/IMPALA-13983
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
I'm reproducing the issue using the master branch at commit
b19331b3d336f680f67c15b4674e2523288d7e59 with an additional sleep before
sending the alter_table HMS RPC in
[CatalogOpExecutor#applyAlterTable()|https://github.com/apache/impala/blob/b19331b3d336f680f67c15b4674e2523288d7e59/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L6731]:
{code:java}
diff --git a/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
b/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
index 4461c03f5..5f5dffb40 100644
--- a/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
+++ b/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
@@ -6728,6 +6728,7 @@ public class CatalogOpExecutor {
MetastoreShim.alterTableWithTransaction(msClient.getHiveClient(),
msTbl, tblTxn);
} else {
try {
+ try {Thread.sleep(20000);} catch (Exception e) {}
msClient.getHiveClient().alter_table(
msTbl.getDbName(), msTbl.getTableName(), msTbl);
} catch (TException e) {{code}
The sleep is added to make the concurrent issue easier to happen.
Steps to reproduce the issue:
{code:sql}
create table tbl (i int);{code}
Run the first AlterTable statement in background:
{code:bash}
impala-shell.sh -q "alter table tbl set tblproperties('k1'='100')" &{code}
Then run another session to submit the Invalidate+AlterTable on the same table,
but setting a new tblproperty:
{code:bash}
impala-shell.sh -q "invalidate metadata tbl; alter table tbl set
tblproperties('k2'='200')"{code}
After both of them finish, describe the table to check the tblproperties.
k1=100 is missing there.
{noformat}
Query: describe formatted tbl
+------------------------------+------------------------------------------------------------+----------------------+
| name | type
| comment |
+------------------------------+------------------------------------------------------------+----------------------+
| # col_name | data_type
| comment |
| | NULL
| NULL |
| i | int
| NULL |
| | NULL
| NULL |
| # Detailed Table Information | NULL
| NULL |
| Database: | default
| NULL |
| OwnerType: | USER
| NULL |
| Owner: | quanlong
| NULL |
| CreateTime: | Tue Apr 22 18:37:00 CST 2025
| NULL |
| LastAccessTime: | UNKNOWN
| NULL |
| Retention: | 0
| NULL |
| Location: | hdfs://localhost:20500/test-warehouse/tbl
| NULL |
| Erasure Coding Policy: | NONE
| NULL |
| Table Type: | EXTERNAL_TABLE
| NULL |
| Table Parameters: | NULL
| NULL |
| | EXTERNAL
| TRUE |
| | OBJCAPABILITIES
| EXTREAD,EXTWRITE |
| | TRANSLATED_TO_EXTERNAL
| TRUE |
| | external.table.purge
| TRUE |
| | k2
| 200 |
| | transient_lastDdlTime
| 1745318281 |
| | NULL
| NULL |
| # Storage Information | NULL
| NULL |
| SerDe Library: |
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
|
| InputFormat: | org.apache.hadoop.mapred.TextInputFormat
| NULL |
| OutputFormat: |
org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
|
| Compressed: | No
| NULL |
| Num Buckets: | 0
| NULL |
| Bucket Columns: | []
| NULL |
| Sort Columns: | []
| NULL |
| | NULL
| NULL |
| # Constraints | NULL
| NULL |
+------------------------------+------------------------------------------------------------+----------------------+{noformat}
CC [~csringhofer] , [~MikaelSmith] , [~rizaon]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
