Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/20707
Change subject: IMPALA-12561: Event-processor shouldn't go into ERROR state for failures in fetching events ...................................................................... IMPALA-12561: Event-processor shouldn't go into ERROR state for failures in fetching events Any failures in fetching HMS events should be retriable. Event-processor should not go into the ERROR state which can only be recovered by a global INVALIDATE METADATA command. This patch deals with the failure when creating a new MetaStoreClient and throws a MetastoreClientInstantiationException instead of an IllegalStateException. Callers can catch the exception and convert it into MetastoreNotificationFetchException which won't fail the event-processor. Also makes sure MetastoreEventsProcessor.getCurrentEventId() only throws MetastoreNotificationFetchException. Previously it throws CatalogException which will fail the event-processor. Note that CatalogException is used for errors in accessing objects in the Catalog, e.g. table not found. We shouldn't throw it when fetching HMS events fails. Tests: - Add test in custom_cluster/test_catalog_hms_failures.py. The test class previously only runs in exhaustive jobs due to long running time. Optimize the test to only restart HMS. Adds a new option, -if_not_running, for run-hive-server.sh to avoid unneccessary restarts. Change-Id: I775684d473fdbfb9f0531234f59a6239bd0873e3 --- M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/MetaStoreClientPool.java A fe/src/main/java/org/apache/impala/catalog/MetastoreClientInstantiationException.java M fe/src/main/java/org/apache/impala/catalog/events/ExternalEventsProcessor.java M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java M fe/src/test/java/org/apache/impala/catalog/events/MetastoreEventsProcessorTest.java M testdata/bin/run-hive-server.sh M tests/custom_cluster/test_catalog_hms_failures.py 8 files changed, 118 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/07/20707/1 -- To view, visit http://gerrit.cloudera.org:8080/20707 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I775684d473fdbfb9f0531234f59a6239bd0873e3 Gerrit-Change-Number: 20707 Gerrit-PatchSet: 1 Gerrit-Owner: Quanlong Huang <[email protected]>
