Quanlong Huang created IMPALA-11490:
---------------------------------------
Summary: More metrics to debug event processing lagging behind
Key: IMPALA-11490
URL: https://issues.apache.org/jira/browse/IMPALA-11490
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Reporter: Quanlong Huang
Event processor could lag behind in many cases, e.g. processing lots events on
large tables, waiting for table locks held by manual refresh or other metadata
operations, etc.
Currently we have metric on the last synced event id. We should also add metric
on the latest event id in HMS. Users can compare them to know whether event
processing is lagging behind.
We should also add logs/metrics on tables that take long time in event
processing, especially those longer than the event polling interval. So users
can decide whether to disable event processing on them, or reduce concurrency
of metadata operations on them.
Some metrics like average events processing duration in the last 5min, 30min or
1h will also be helpful.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
