[
https://issues.apache.org/jira/browse/IMPALA-14684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18102155#comment-18102155
]
ASF subversion and git services commented on IMPALA-14684:
----------------------------------------------------------
Commit 538ceb4223d1ceffb525c6fa355de3d459d6c585 in impala's branch
refs/heads/master from Arnab Karmakar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=538ceb422 ]
IMPALA-14684: Add metrics for catalogd table invalidations
Implements sliding window metrics to track table invalidations due to
TTL and memory pressure, helping administrators monitor and tune
configurations like invalidate_tables_timeout_s and JVM heap size.
New metrics added:
- catalog.num-ttl-invalidated-tables: Total TTL-based invalidations
- catalog.num-memory-pressure-invalidated-tables: Total memory pressure
invalidations
- catalog.ttl-invalidations-{10s,01m,05m,30m}: TTL invalidations in
sliding windows
- catalog.memory-pressure-invalidations-{10s,01m,05m,30m}: Memory
pressure invalidations in sliding windows
- catalog.last-ttl-invalidated-tables: Tables invalidated by last TTL
invalidation batch
- catalog.last-ttl-invalidation-ms: Milliseconds since epoch of last TTL
invalidation batch
- catalog.last-memory-pressure-invalidated-tables: Tables invalidated by
last memory pressure invalidation batch
- catalog.last-memory-pressure-invalidation-ms: Milliseconds since epoch
of last memory pressure invalidation batch
Changes:
- CatalogdTableInvalidator tracks timestamps, Catalog provides counters
- Added 14 new fields to TGetCatalogServerMetricsResponse
- Updated CatalogServer to register and refresh new metrics
Testing:
- Added JUnit tests that verify timestamp queue operations and binary
search efficiency
- Added integration test that validates TTL-based invalidation and sliding
window metrics
Change-Id: If78e6c8429801e3c1829860666f04ad287b826ab
Assisted-by: composer-2.5-fast
Reviewed-on: http://gerrit.cloudera.org:8080/24166
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add metrics for CatalogdTableInvalidator
> ----------------------------------------
>
> Key: IMPALA-14684
> URL: https://issues.apache.org/jira/browse/IMPALA-14684
> Project: IMPALA
> Issue Type: New Feature
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Arnab Karmakar
> Priority: Critical
>
> We need catalogd metrics to show how many tables are invalidated due to TTL
> and memory pressure. Also add metrics to show the rate of them, e.g. numbers
> in the past 1s/1m/5m/30m, etc.
> These help admins to adjust configurations like invalidate_tables_timeout_s
> and the JVM heap size of catalogd.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
