Quanlong Huang created IMPALA-9670:
--------------------------------------
Summary: Fix unloaded views are shown as tables for GET_TABLES
requests
Key: IMPALA-9670
URL: https://issues.apache.org/jira/browse/IMPALA-9670
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Quanlong Huang
JDBC/ODBC clients will send GET_TABLES requests to get the list of tables of a
given db. Table types and comments should be returned as well. However, for
unloaded views which are represended as IncompleteTable, Impala doesn't know
the exact table types and return them as tables by default. This causes some
troubles for applications that needs to distinguish views.
Problems to fix in this JIRA:
* Views are shown as tables when Impala launches (all views are unloaded).
* Views are shown as tables after creation (the created view is unloaded).
Possible solution is adding table type to IncompleteTable:
* _IncompleteTable_s are created in several places. Add table type in those
places.
* For startup and global invalidate metadata,
** Approach 1: use getTables API with table type.
*** currently we use IMetaStoreClient#getAllTables(dbName) to get table names
inside a db and create IncompleteTables for them.
*** We can add an additional call to load names of views using
IMetaStoreClient#getTables(dbName, tablePattern, tableType).
*** This may increase the startup time by 2x.
** Approach 2: use getTableMeta API instead of getAllTables.
*** Pros: can get the comments as well.
*** Cons: This may increase the startup time by 4x.
* Load view's metadata at the end of lightweight DDL operations like
"INVALIDATE METADATA _table_", "CREATE VIEW", etc.
* EventProcessors can extract table type from the event's HMS table object.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
