Todd Lipcon has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/10713 )
Change subject: IMPALA-7140 (part 3): load partitions for FS tables ...................................................................... IMPALA-7140 (part 3): load partitions for FS tables This implements the loading of partitions from the HMS for FS-based tables. The LocalPartitionSpec class implements PrunablePartition based on parsing the partition names returned from the HMS. After partitions are pruned, the remaining partitions can be loaded by name. With this patch, I am able to connect to an impalad running with --use_local_catalog and issue 'show partitions functional.alltypes' with the expected results. A new unit test is added which shares some code with CatalogTest to ensure that partitions can be loaded and parsed properly. Testing partition pruning via end-to-end planning is not quite supported yet: we need to implement 'toThriftDescriptor()' before we can run those end-to-end tests. Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3 Reviewed-on: http://gerrit.cloudera.org:8080/10713 Reviewed-by: Vuk Ercegovac <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java A fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java A fe/src/main/java/org/apache/impala/catalog/local/LocalPartitionSpec.java M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java 14 files changed, 707 insertions(+), 108 deletions(-) Approvals: Vuk Ercegovac: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/10713 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3 Gerrit-Change-Number: 10713 Gerrit-PatchSet: 7 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tianyi Wang <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]>
