[
https://issues.apache.org/jira/browse/IMPALA-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18062604#comment-18062604
]
Quanlong Huang commented on IMPALA-14684:
-----------------------------------------
Invalidations are lightweight operations (especially for implicit
invalidations) that might only take milliseconds. So the number for the past 1s
is still helpful, especially when we want to match them with JVM GC events.
Metrics collector can collect this counter every second so we can see the burst
in monitoring charts.
For the implementation, we don't need avg/min/max metrics so it can be much
simpler. I think we just need a queue to collect all the timestamps of the
recent invalidations. Bisearch can efficiently count how many timestamps are
larger than a given timestamp. Timestamps older than 30m can be removed from
the queue.
> Add metrics for CatalogdTableInvalidator
> ----------------------------------------
>
> Key: IMPALA-14684
> URL: https://issues.apache.org/jira/browse/IMPALA-14684
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Arnab Karmakar
> Priority: Critical
>
> We need catalogd metrics to show how many tables are invalidated due to TTL
> and memory pressure. Also add metrics to show the rate of them, e.g. numbers
> in the past 1s/1m/5m/30m, etc.
> These help admins to adjust configurations like invalidate_tables_timeout_s
> and the JVM heap size of catalogd.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
