[
https://issues.apache.org/jira/browse/IMPALA-14230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18050359#comment-18050359
]
Arnab Karmakar commented on IMPALA-14230:
-----------------------------------------
[~stigahuang] I can take up this if you feel I can handle it. After having
worked on IMPALA-13491 , I have a better understanding on how to add flags.
Moreover, we'll need some changes in *MetastoreEventsProcessor.java* to detect
when to enter catch-up mode using the metrics latestEventId, lastSyncedEventId
and I think we'll need some modifications in
*CatalogServer#WaitUntilHmsEventsSynced()* to enable catch-up mode during
failover.
We could discuss about the design in depth if you feel like assigning it to me.
> Add a "catch-up" mode to EventProcessor
> ---------------------------------------
>
> Key: IMPALA-14230
> URL: https://issues.apache.org/jira/browse/IMPALA-14230
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Reporter: Quanlong Huang
> Priority: Critical
>
> When HMS events processing has a large lag, we need EventProcessor to quickly
> catch up with HMS by simply processing the events as invalidations, i.e.
> falling back to the behavior before IMPALA-9101 that table level events just
> invalidate the table.
> This will also be useful in catalogd HA failover that the passive catalogd
> wants to quickly catch up with HMS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
