Robbie Zhang created IMPALA-10082:
-------------------------------------
Summary: Concurrent invalidate metadata and create/drop table
cause discrepancy in metadata
Key: IMPALA-10082
URL: https://issues.apache.org/jira/browse/IMPALA-10082
Project: IMPALA
Issue Type: Bug
Affects Versions: Impala 4.0
Reporter: Robbie Zhang
The symptom is similar to IMPALA-7093 but is a different issue. Here is how I
reproduce it:
1) Ran the first script to keep running create/insert/drop queries
{code:java}
#!/bin/bash
while [ 1 ]
do
shell/impala-shell -q "create table if not exists test(i int); insert into
test(i) values(1); drop table test;" 2>&1| tee test.output
n=`egrep "Exception" test.output | wc -l`
if [ $n -lt 0 ]; then
rm -f /tmp/testing
exit
fi
done
{code}
2) Ran the second script to keep running global invalidate metadata
{code:java}
#!/bin/bash
while [ 1 ]
do
shell/impala-shell -q "invalidate metadata"
done
{code}
Sometime later, the first scrip ended with "Table default.test does not exist":
{code:java}
Starting Impala Shell with no authentication using Python 2.7.12
Warning: live_progress only applies to interactive shell sessions, and is being
skipped for now.
Opened TCP connection to localhost:21000
Connected to localhost:21000
Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build
f95f7940e4a290d75ee85fd78e85bc26795f0f9f)
Query: create table if not exists test(i int)
Fetched 1 row(s) in 0.01s
Query: insert into test(i) values(1)
Query submitted at: 2020-08-13 22:57:51 (Coordinator: http://impala34:25000)
ERROR: AnalysisException: Table does not exist: default.test
Could not execute command: insert into test(i) values(1){code}
Even after I change to local catalog mode, this issue still exists:
{code:java}
Starting Impala Shell with no authentication using Python 2.7.12
Warning: live_progress only applies to interactive shell sessions, and is being
skipped for now.
Opened TCP connection to localhost:21000
Connected to localhost:21000
Server version: impalad version 4.0.0-SNAPSHOT DEBUG (build
f95f7940e4a290d75ee85fd78e85bc26795f0f9f)
Query: create table if not exists test(i int)
Fetched 1 row(s) in 0.07s
Query: insert into test(i) values(1)
Query submitted at: 2020-08-13 22:10:16 (Coordinator: http://impala34:25000)
ERROR: AnalysisException: org.apache.impala.catalog.TableLoadingException:
Could not load table default.test from catalog
CAUSED BY: TableLoadingException: Could not load table default.test from catalog
CAUSED BY: TException:
TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL,
error_msgs:[TableLoadingException: Table default.test no longer exists in the
Hive MetaStore. Run 'invalidate metadata default.test' to update the Impala
catalog.]), lookup_status:OK)
Could not execute command: insert into test(i) values(1)
{code}
And in local catalog mode, the newly created table was lost but it's still
visible in the coordinator. After running 'invalidate metadata default.test',
the table disappeared at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
