Quanlong Huang created IMPALA-14307:
---------------------------------------
Summary: invalidateTable could add table to old Db object while
concurrent reset is running
Key: IMPALA-14307
URL: https://issues.apache.org/jira/browse/IMPALA-14307
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Saw a failure in TestConcurrentDdls.test_ddls_with_invalidate_metadata:
{code:python}
custom_cluster/test_concurrent_ddls.py:50: in test_ddls_with_invalidate_metadata
self._run_ddls_with_invalidation(unique_database, sync_ddl=False)
custom_cluster/test_concurrent_ddls.py:179: in _run_ddls_with_invalidation
worker[i].get(timeout=100)
/data/jenkins/workspace/impala-asf-master-core-ubsan-arm/Impala-Toolchain/toolchain-packages-gcc10.4.0/python-2.7.16/lib/python2.7/multiprocessing/pool.py:572:
in get
raise self._value
E AssertionError: Query b148f272b5821772:ace4369900000000 failed:
E AnalysisException: Table does not exist:
test_ddls_with_invalidate_metadata_9525e717.test_6
E
E
E assert <bound method type.is_acceptable_error of <class
'test_concurrent_ddls.TestConcurrentDdls'>>('Query
b148f272b5821772:ace4369900000000 failed:\nAnalysisException: Table does not
exist: test_ddls_with_invalidate_metadata_9525e717.test_6\n\n', False)
E + where <bound method type.is_acceptable_error of <class
'test_concurrent_ddls.TestConcurrentDdls'>> =
<test_concurrent_ddls.TestConcurrentDdls object at
0xffff1ff7d810>.is_acceptable_error{code}
The test runs in legacy catalog mode. Coordinator logs show that it receives a
deletion update on the table before analying the REFRESH statement on it:
{noformat}
I20250811 16:22:58.429529 3608823 ImpaladCatalog.java:227] Deleting:
TABLE:test_ddls_with_invalidate_metadata_9525e717.test_6 version: 8523 size: 97
...
I20250811 16:22:58.540377 3609121 Frontend.java:2398]
b148f272b5821772:ace4369900000000] Analyzing query: refresh
test_ddls_with_invalidate_metadata_9525e717.test_6 db: default
...
I20250811 16:22:58.542122 3609121 jni-util.cc:321]
b148f272b5821772:ace4369900000000] org.apache.impala.common.AnalysisException:
Table does not exist: test_ddls_with_invalidate_metadata_9525e717.test_6
at
org.apache.impala.analysis.ResetMetadataStmt.analyze(ResetMetadataStmt.java:173)
at
org.apache.impala.analysis.AnalysisContext$AnalysisDriverImpl.analyze(AnalysisContext.java:576)
at
org.apache.impala.analysis.AnalysisContext.analyzeAndAuthorize(AnalysisContext.java:496)
at
org.apache.impala.service.Frontend.doCreateExecRequest(Frontend.java:2938)
at
org.apache.impala.service.Frontend.getTExecRequest(Frontend.java:2505)
at
org.apache.impala.service.Frontend.getTExecRequestWithFallback(Frontend.java:2374)
at
org.apache.impala.service.Frontend.createExecRequest(Frontend.java:2063)
at
org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:176){noformat}
Catalogd logs show that invalidateTable triggered by alterTableOrViewRename()
is running concurrently with a catalog reset (i.e. global INVALIDATE METADATA):
{noformat}
I20250811 16:22:58.129398 3609170 JniUtil.java:167]
f548d67b72570890:626780e200000000] execDdl request: ALTER_TABLE
test_ddls_with_invalidate_metadata_9525e717.test_6_2 RENAME_TABLE issued by
jenkins
I20250811 16:22:58.144670 3608935 JniUtil.java:167]
1d4918bff853c0ec:c5646e4a00000000] resetMetadata request: INVALIDATE ALL issued
by jenkins
I20250811 16:22:58.173722 3608935 CatalogServiceCatalog.java:2339]
1d4918bff853c0ec:c5646e4a00000000] Loaded table list (12) and functions (0) for
database: test_ddls_with_invalidate_metadata_9525e717. Fetch duration: 2 ms.
I20250811 16:22:58.177870 3609170 CatalogServiceCatalog.java:3283]
f548d67b72570890:626780e200000000] Invalidating table metadata:
test_ddls_with_invalidate_metadata_9525e717.test_6
I20250811 16:22:58.178402 3608935 JniUtil.java:178]
1d4918bff853c0ec:c5646e4a00000000] Finished resetMetadata request: INVALIDATE
ALL issued by jenkins. Time spent: 34ms
I20250811 16:22:58.181602 3609170 Table.java:246]
f548d67b72570890:626780e200000000] createEventId_ for table:
test_ddls_with_invalidate_metadata_9525e717.test_6 set to: -1
I20250811 16:22:58.181922 3609170 JniUtil.java:178]
f548d67b72570890:626780e200000000] Finished execDdl request: ALTER_TABLE
test_ddls_with_invalidate_metadata_9525e717.test_6_2 RENAME_TABLE issued by
jenkins. Time spent: 52ms
{noformat}
Then the catalog update collector thread saw the table disappears and sent the
deletion update:
{noformat}
I20250811 16:22:58.247798 3608258 JniUtil.java:167] getCatalogDelta request:
Getting catalog delta from version 8517
I20250811 16:22:58.250487 3608258 catalog-server.cc:1615] Collected deletion:
1:TABLE:test_ddls_with_invalidate_metadata_9525e717.test_6, version=8523,
original size=97, compressed size=95
I20250811 16:22:58.250890 3608258 catalog-server.cc:1615] Collected update:
1:CATALOG_SERVICE_ID, version=8532, original size=60, compressed size=58
I20250811 16:22:58.250959 3608258 JniUtil.java:178] Finished getCatalogDelta
request: Getting catalog delta from version 8517. Time spent: 3ms{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
