Hello Impala Public Jenkins, Vuk Ercegovac,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/10629
to look at the new patch set (#3).
Change subject: IMPALA-7137. Support configuring Frontend to use LocalCatalog
......................................................................
IMPALA-7137. Support configuring Frontend to use LocalCatalog
This adds a new flag -use_local_catalog which is passed through to the
frontend and causes it to use LocalCatalog instead of ImpaladCatalog.
Additionally, when this flag is configured, the impalad does not
subscribe to catalog topic updates from the statestore.
The patch is slightly more complex than simply picking which class to
instantiate, because the lifecycle is designed a bit differently between
the two implementations:
- LocalCatalog is instantiated once per query/request.
- ImpaladCatalog is instantiated once and stateful across queries,
except when a full catalog update is received. This maintains the
current behavior for this implementation.
In order to abstract this difference in lifecycle from the frontend, I
introduced a new FeCatalogManager class with different implementations
for the two lifecycles. I also had to add a simple test implementation
since some tests rely on directly injecting a Catalog implementation
into the Frontend.
This patch also includes a few small changes to the local catalog
implementation objects to enable the impalad to start and accept
connections. With this patch, I was able to manually test as follows:
I started just the statestore and the impalad in the new mode:
- ./bin/start-statestored.sh
- ./bin/start-impalad.sh --use_local_catalog
I connected with impala-shell as usual and was able to run the most
simple queries:
- SHOW DATABASES;
- USE functional;
- SHOW TABLES;
All other functionality results in error messages due to the various
TODOs in the current skeleton implementation.
Change-Id: I8c9665bd031d23608740b23eef301970af9aa764
---
M be/src/runtime/exec-env.cc
M be/src/service/impala-server.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java
M fe/src/main/java/org/apache/impala/service/DescribeResultFactory.java
A fe/src/main/java/org/apache/impala/service/FeCatalogManager.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/JniFrontend.java
M fe/src/main/java/org/apache/impala/service/MetadataOp.java
11 files changed, 238 insertions(+), 63 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/10629/3
--
To view, visit http://gerrit.cloudera.org:8080/10629
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8c9665bd031d23608740b23eef301970af9aa764
Gerrit-Change-Number: 10629
Gerrit-PatchSet: 3
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com>
