Hello Quanlong Huang, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/24166
to look at the new patch set (#7).
Change subject: IMPALA-14684: Add metrics for catalogd table invalidations
......................................................................
IMPALA-14684: Add metrics for catalogd table invalidations
Implements sliding window metrics to track table invalidations due to
TTL and memory pressure, helping administrators monitor and tune
configurations like invalidate_tables_timeout_s and JVM heap size.
New metrics added:
- catalog.num-ttl-invalidated-tables: Total TTL-based invalidations
- catalog.num-memory-pressure-invalidated-tables: Total memory pressure
invalidations
- catalog.ttl-invalidations-{01s,01m,05m,30m}: TTL invalidations in
sliding windows
- catalog.memory-pressure-invalidations-{01s,01m,05m,30m}: Memory
pressure invalidations in sliding windows
- catalog.last-ttl-invalidated-tables: Tables invalidated by last TTL
invalidation batch
- catalog.last-ttl-invalidation-ms: Milliseconds since epoch of last TTL
invalidation batch
- catalog.last-memory-pressure-invalidated-tables: Tables invalidated by
last memory pressure invalidation batch
- catalog.last-memory-pressure-invalidation-ms: Milliseconds since epoch
of last memory pressure invalidation batch
Changes:
- CatalogdTableInvalidator tracks timestamps, Catalog provides counters
- Added 14 new fields to TGetCatalogServerMetricsResponse
- Updated CatalogServer to register and refresh new metrics
Testing:
- Added JUnit tests that verify timestamp queue operations and binary
search efficiency
- Added integration test that validates TTL-based invalidation and sliding
window metrics
Change-Id: If78e6c8429801e3c1829860666f04ad287b826ab
Assisted-by: composer-2
---
M be/src/catalog/catalog-server.cc
M be/src/catalog/catalog-server.h
M common/thrift/JniCatalog.thrift
M common/thrift/metrics.json
M fe/src/main/java/org/apache/impala/catalog/Catalog.java
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/CatalogdTableInvalidator.java
M fe/src/main/java/org/apache/impala/service/JniCatalog.java
M fe/src/test/java/org/apache/impala/catalog/CatalogdTableInvalidatorTest.java
M tests/custom_cluster/test_automatic_invalidation.py
10 files changed, 812 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/66/24166/7
--
To view, visit http://gerrit.cloudera.org:8080/24166
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If78e6c8429801e3c1829860666f04ad287b826ab
Gerrit-Change-Number: 24166
Gerrit-PatchSet: 7
Gerrit-Owner: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Arnab Karmakar <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
