Quanlong Huang created IMPALA-13116:
---------------------------------------
Summary: In local-catalog mode, abort REFRESH and metadata
reloading of DDL/DMLs if the table is invalidated
Key: IMPALA-13116
URL: https://issues.apache.org/jira/browse/IMPALA-13116
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Quanlong Huang
A table can be invalidated when there are DDL/DML/REFRESHs running in flight:
* User can explictly trigger an INVALIDATE METADATA <table> command
* The table could be invalidated by CatalogdTableInvalidator when
invalidate_tables_on_memory_pressure or invalidate_tables_timeout_s is turned on
Note that invalidating a table doesn't require holding the lock of the
HdfsTable object so it can finish even if there are on-going updates on the
table.
The updated HdfsTable object won't be added to the metadata cache since it has
been replaced with an IncompleteTable object. It's only used in the
DDL/DML/REFRESH responses. In local catalog mode, the response is the minimal
representation which is mostly the table name and catalog version. We don't
need the updates on the HdfsTable object to be finished. Thus, we can consider
aborting the reloading of such DDL/DML/REFRESH requests.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
