Vuk Ercegovac has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11608
Change subject: IMPALA-7597: wraps retries around CatalogMetaProvider ...................................................................... IMPALA-7597: wraps retries around CatalogMetaProvider When configured to use the local catalog, concurrent metadata reads and writes can cause the CatalogMetaProvider to throw an InconsistentMetadataFetchException. Queries have been wrapped with a retry loop, but the other frontend methods, such listing table or partition information, can also fail from the same error. These errors were seen under a workload consisting of concurrent adding and showing partitions. This change adds a RetryableMetaProvider that wraps the relevant methods of CatalogMetaProvider with retries/logging. When obtaining a Catalog from the FeCatalogManager, the caller can switch between obtaining a Catalog that does not retry and one that does. Several Frontend call-sites have been updated accordingly. For example, GetCatalogMetrics does not really care if an exception is thrown when fetching the number of tables so using a Catalog without retries is used (as before). However, when showing various metadata such as partitions or files, a retryable Catalog is now used. Note that when a local catalog is not used, the retryable catalog is a no-op. Testing: - added a test that checks whether inconsistent metadata exceptions are seen in a concurrent workload. Change-Id: I43a21571d54a7966c5c68bea1fe69dbc62be2a0b --- A fe/src/main/java/org/apache/impala/catalog/local/RetryableMetaProvider.java M fe/src/main/java/org/apache/impala/service/FeCatalogManager.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/MetadataOp.java M tests/custom_cluster/test_local_catalog.py 5 files changed, 323 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/08/11608/1 -- To view, visit http://gerrit.cloudera.org:8080/11608 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I43a21571d54a7966c5c68bea1fe69dbc62be2a0b Gerrit-Change-Number: 11608 Gerrit-PatchSet: 1 Gerrit-Owner: Vuk Ercegovac <[email protected]>
