Csaba Ringhofer created IMPALA-12474:
----------------------------------------
Summary: Latest metastore event id polling can be blocked
Key: IMPALA-12474
URL: https://issues.apache.org/jira/browse/IMPALA-12474
Project: IMPALA
Issue Type: Bug
Components: Catalog
Reporter: Csaba Ringhofer
Latest event id polling is supposed to work independently from event batch
polling:
https://github.com/apache/impala/blob/2d3289027c2ffdd245d13b60e6fa3f9b3e7bf833/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L684
This doesn't work in reality as the scheduler is single threaded:
https://github.com/apache/impala/blob/2d3289027c2ffdd245d13b60e6fa3f9b3e7bf833/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L533C44-L533C44
A related potential improvement: scheduleWithFixedDelay adds a delay after
batch processing, even it was slow - if Impala know that it is lagging behind
then it should start to work on the next batch immediately.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
