Anurag Mantripragada has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/14731 )
Change subject: IMPALA-9158: Support loading primary key/foreign key constraints in LocalCatalog Mode. ...................................................................... IMPALA-9158: Support loading primary key/foreign key constraints in LocalCatalog Mode. This change add a new method 'loadConstraints()' to the MetaProvider interface. 1. In CatalogdMetaProvider implementation, we fetch the primary key (PK) and foreign key(FK) information via the GetPartialCatalogObject() RPC to the catalogd. This is modified to include PK/FK information. This is because, the V1 implementation eagerly loads PK/FK information and hence can be sent over without making any additional RPCs to HMS. This information is then stored in TableMetaRef object for future consumers to directly get this information. This way, this information is only valid as long as the table cache is valid. 2. In the DirectMetaProvider implementation, we make two RPCs to HMS to directly get PK/FK information. Load constraints can be extended to include other constraints later (for ex: unique constraints.) Testing: - Added tests in LocalCatalogTest, CatalogTest and PartialCatalogInfoTest Change-Id: I7ea7e1bacf6eb502c67caf310a847b32687e0d58 --- M bin/rat_exclude_files.txt M common/thrift/CatalogService.thrift M fe/src/main/java/org/apache/impala/analysis/TableDef.java M fe/src/main/java/org/apache/impala/analysis/ToSqlUtils.java M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/PartialCatalogInfoTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java A testdata/data/child_table.txt A testdata/data/parent_table.txt A testdata/data/parent_table_2.txt M testdata/datasets/functional/functional_schema_template.sql 17 files changed, 412 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/14731/3 -- To view, visit http://gerrit.cloudera.org:8080/14731 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7ea7e1bacf6eb502c67caf310a847b32687e0d58 Gerrit-Change-Number: 14731 Gerrit-PatchSet: 3 Gerrit-Owner: Anurag Mantripragada <anu...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com>
