[
https://issues.apache.org/jira/browse/IMPALA-12832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824833#comment-17824833
]
ASF subversion and git services commented on IMPALA-12832:
----------------------------------------------------------
Commit b7ddbcad0dd6accb559a3f391a897a8c442d1728 in impala's branch
refs/heads/master from Venu Reddy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b7ddbcad0 ]
IMPALA-12832: Implicit invalidate metadata on event failures
At present, failure in event processing needs manual invalidate
metadata. This patch implicitly invalidates the table upon failures
in processing of table events with new
'invalidate_metadata_on_event_processing_failure' flag. And a new
'invalidate_global_metadata_on_event_processing_failure' flag is
added to global invalidate metadata automatically when event
processor goes to non-active state.
Note: Also introduced a config
'inject_process_event_failure_event_types' for automated tests to
simulate event processor failures. This config is used to specify what
event types can be intentionally failed. This config should only be
used for testing purpose. Need IMPALA-12851 as a prerequisite
Testing:
- Added end-to-end tests to mimic failures in event processor and verified
that event processor is active
- Added unit test to verify the 'auto_global_invalidate_metadata' config
- Passed FE tests
Co-Authored-by: Sai Hemanth Gantasala <[email protected]>
Change-Id: Ia67fc04c995802d3b6b56f79564bf0954b012c6c
Reviewed-on: http://gerrit.cloudera.org:8080/21065
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> EventProcessor shouldn't stop for failures in single-table event
> ----------------------------------------------------------------
>
> Key: IMPALA-12832
> URL: https://issues.apache.org/jira/browse/IMPALA-12832
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Venugopal Reddy K
> Priority: Critical
> Fix For: Impala 4.4.0
>
>
> EventProcessor goes into the ERROR/NEEDS_INVALIDATE state when it hits
> unexpected failures in processing an event. The cause are usually bugs. When
> it stops, all tables that need sync will be impacted. However, the event
> might just a single-table event (in contrast to multi-table events like
> RenameTable, DropDatabaseCascade, CommitTxn, AbortTxn). We can consider
> skipping the event and just invalidating the table.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
