Quanlong Huang created IMPALA-12865:
---------------------------------------
Summary: enable_reload_events breaks enable_skipping_older_events
by pushing lastRefreshEventId too high
Key: IMPALA-12865
URL: https://issues.apache.org/jira/browse/IMPALA-12865
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Quanlong Huang
Assignee: Sai Hemanth Gantasala
When enable_reload_events is true, catalogd fires RELOAD events after REFRESH
finishes reloading the table/partition. The RELOAD event id is also used to
update lastRefreshEventId of the table/partition.
[https://github.com/apache/impala/blob/784971c018c2dc44c53d7c0f366ad49cd8681ac6/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L6807-L6814]
This is problematic when enable_skipping_older_events is true. HMS events
generated after the reload and before the RELOAD event will be skipped.
[https://github.com/apache/impala/blob/784971c018c2dc44c53d7c0f366ad49cd8681ac6/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L1219-L1221]
The issue is identified in the review of
[https://gerrit.cloudera.org/c/21096/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#6799]
CC [~hemanth619], [~VenuReddy], [~csringhofer]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
