[
https://issues.apache.org/jira/browse/IMPALA-11535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17698825#comment-17698825
]
Sai Hemanth Gantasala commented on IMPALA-11535:
------------------------------------------------
With the Implementation of IMPALA-11822, We can skip some of the events by
comparing the current event's id with the latestRefreshEventId of that
table/partition.
> Skip events happen before manual REFRESH
> ----------------------------------------
>
> Key: IMPALA-11535
> URL: https://issues.apache.org/jira/browse/IMPALA-11535
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Quanlong Huang
> Assignee: Sai Hemanth Gantasala
> Priority: Critical
>
> If the table has been manually refreshed, all its events happen before the
> manual REFRESH can be skipped.
>
> This happens when catalogd is lagging behind in processing events. When
> processing an event, we can check whether there are manual REFRESH executed
> after its eventTime. In such case, we don't need to process the event to
> refresh anything. This helps catalogd to catch up HMS events quickly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
