Hello Quanlong Huang, Sai Hemanth Gantasala, Csaba Ringhofer, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21065
to look at the new patch set (#9).
Change subject: [WIP]IMPALA-12832: Implicit invalidate metadata on event
failures
......................................................................
[WIP]IMPALA-12832: Implicit invalidate metadata on event failures
At present, failure in event processing needs manual invalidate
metadata. This patch implicitly invalidates the table upon failures
in processing of table events with new 'process_event_failure' flag.
And a new 'auto_global_invalidate_metadata' flag is added to global
invalidate metadata automatically when event processor goes to
non-active state.
Note: Also introduced a config
'inject_process_event_failure_event_types' for automated tests to
simulate event processor failures. This config is used to specify what
event types can be intentionally failed. This config should only be
used for testing purpose.
Testing:
- Added end-to-end tests to mimic failures in event processor and verified
that event processor is active
- Passed FE tests
Co-Authored-by: Sai Hemanth Gantasala <[email protected]>
Change-Id: Ia67fc04c995802d3b6b56f79564bf0954b012c6c
---
M be/src/catalog/catalog-server.cc
M be/src/util/backend-gflag-util.cc
M common/thrift/BackendGflags.thrift
M fe/src/compat-apache-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java
M
fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java
M fe/src/main/java/org/apache/impala/service/BackendConfig.java
M tests/common/impala_test_suite.py
A tests/metadata/__init__.py
M tests/metadata/test_event_processing.py
A tests/metadata/test_event_processing_base.py
A tests/metadata/test_event_processing_error.py
M tests/util/event_processor_utils.py
15 files changed, 664 insertions(+), 304 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/21065/9
--
To view, visit http://gerrit.cloudera.org:8080/21065
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia67fc04c995802d3b6b56f79564bf0954b012c6c
Gerrit-Change-Number: 21065
Gerrit-PatchSet: 9
Gerrit-Owner: Anonymous Coward <[email protected]>
Gerrit-Reviewer: Anonymous Coward <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]>
