Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15887 )
Change subject: IMPALA-9669: Fix wrong types/comments of loaded tables/views for GET_TABLES in LocalCatalog ...................................................................... IMPALA-9669: Fix wrong types/comments of loaded tables/views for GET_TABLES in LocalCatalog Coordinator can be in two modes: legacy mode or LocalCatalog mode. Before IMPALA-8606, GET_TABLES required all tables to be loaded in LocalCatalog-mode coordinator’s cache, which is a performance regression compared to legacy mode coordinators. IMPALA-8606 changes the behavior to only load the table names and create LocalIncompleteTable for each table, which boosts the performance but results in all views being returned with the default table type (TABLE). Besides this, all returned comments are empty even if the table/view is loaded. This is a regression since in legacy coordinators, loaded tables/views are shown with correct table types and comments. This is fixed by loading table types and comments along with table names from catalogd. The cached list of table names of a DB is changed to be a map containing the brief table metadata (name, type, comment). In case of stale types or comments in the list, when loading the msTable of a table, coordinator checks the type and comment and invalidates the table list if any of them are stale. Tests - Add tests in test_hs2.test_get_tables and manually test it in LocalCatalog mode. - Run CORE tests. Change-Id: I2180c603f061838347936f718cd4a0257d82e633 Reviewed-on: http://gerrit.cloudera.org:8080/15887 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M common/thrift/CatalogService.thrift M fe/src/main/java/org/apache/impala/catalog/Db.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIncompleteTable.java M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java M fe/src/main/java/org/apache/impala/service/MetadataOp.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M fe/src/test/java/org/apache/impala/catalog/local/CatalogdMetaProviderTest.java M tests/hs2/test_hs2.py M tests/query_test/test_observability.py 12 files changed, 249 insertions(+), 78 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15887 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I2180c603f061838347936f718cd4a0257d82e633 Gerrit-Change-Number: 15887 Gerrit-PatchSet: 11 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Anurag Mantripragada <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
