Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11608
to look at the new patch set (#2).
Change subject: IMPALA-7597: wraps retries around CatalogMetaProvider
......................................................................
IMPALA-7597: wraps retries around CatalogMetaProvider
When configured to use the local catalog, concurrent metadata
reads and writes can cause the CatalogMetaProvider to throw
an InconsistentMetadataFetchException. Queries have been wrapped
with a retry loop, but the other frontend methods, such listing
table or partition information, can also fail from the same error.
These errors were seen under a workload consisting of concurrent
adding and showing partitions.
This change adds a RetryableMetaProvider that wraps the relevant
methods of CatalogMetaProvider with retries/logging. When obtaining
a Catalog from the FeCatalogManager, the caller can switch between
obtaining a Catalog that does not retry and one that does. Several
Frontend call-sites have been updated accordingly. For example,
GetCatalogMetrics does not really care if an exception is thrown
when fetching the number of tables so using a Catalog without retries
is used (as before). However, when showing various metadata such as
partitions or files, a retryable Catalog is now used. Note that when
a local catalog is not used, the retryable catalog is a no-op.
Testing:
- added a test that checks whether inconsistent metadata exceptions
are seen in a concurrent workload.
Change-Id: I43a21571d54a7966c5c68bea1fe69dbc62be2a0b
---
A fe/src/main/java/org/apache/impala/catalog/local/RetryableMetaProvider.java
M fe/src/main/java/org/apache/impala/service/FeCatalogManager.java
M fe/src/main/java/org/apache/impala/service/Frontend.java
M fe/src/main/java/org/apache/impala/service/MetadataOp.java
M tests/custom_cluster/test_local_catalog.py
5 files changed, 328 insertions(+), 44 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/11608/2
--
To view, visit http://gerrit.cloudera.org:8080/11608
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I43a21571d54a7966c5c68bea1fe69dbc62be2a0b
Gerrit-Change-Number: 11608
Gerrit-PatchSet: 2
Gerrit-Owner: Vuk Ercegovac <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
