Hello Tianyi Wang, Vuk Ercegovac,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10713
to look at the new patch set (#4).
Change subject: IMPALA-7140 (part 3): load partitions for FS tables
......................................................................
IMPALA-7140 (part 3): load partitions for FS tables
This implements the loading of partitions from the HMS for FS-based
tables. The LocalPartitionSpec class implements PrunablePartition based
on parsing the partition names returned from the HMS. After partitions
are pruned, the remaining partitions can be loaded by name.
With this patch, I am able to connect to an impalad running with
--use_local_catalog and issue 'show partitions functional.alltypes' with
the expected results.
A new unit test is added which shares some code with CatalogTest to
ensure that partitions can be loaded and parsed properly.
Testing partition pruning via end-to-end planning is not quite
supported yet: we need to implement 'toThriftDescriptor()' before we can
run those end-to-end tests.
Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3
---
M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java
M fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java
A fe/src/main/java/org/apache/impala/catalog/local/LocalFsPartition.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
A fe/src/main/java/org/apache/impala/catalog/local/LocalPartitionSpec.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java
14 files changed, 707 insertions(+), 108 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/13/10713/4
--
To view, visit http://gerrit.cloudera.org:8080/10713
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Iddf2edbd6bdc0684560b2ecca9c4c6b6819ef1d3
Gerrit-Change-Number: 10713
Gerrit-PatchSet: 4
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com>
