[
https://issues.apache.org/jira/browse/HDDS-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-14993:
-----------------------------------
Description:
h2. Completed scope
HDDS-15256, implemented by [PR
#10253|https://github.com/apache/ozone/pull/10253] and merged on 18 May 2026,
added the Snapshot Defrag Metrics panels originally requested here. It includes
overall/full/incremental defrag counts and failures, skipped snapshots, tables
compacted, and delta files processed. These panels do not need to be
implemented again.
The PR did not add SSTFilePruningMetrics or the four DeletingServiceMetrics
snapshot gauges listed below; those remain the scope of this issue.
h2. Remaining scope: SST pruning and snapshot deletion metrics in Grafana
Add the remaining observability panels to the existing dashboard:
# SSTFilePruningMetrics: filesPrunedTotal, filesPrunedLast, filesSkippedTotal,
compactionsProcessed, pruneQueueSize, and pruningFailures.
# DeletingServiceMetrics snapshot gauges: snapKeysReclaimedLast,
snapReclaimedSizeLast, snapKeysIteratedLast, and snapKeysNotReclaimableLast.
Use the actual exported Prometheus metric names and labels from these metric
sources. Include appropriate units, descriptions, and OM-instance filtering,
following existing dashboard conventions. Verify the panels against exported
metrics so queries refer to real series.
h2. Existing implementation and metric sources
Repository baseline: d0949f291791c399f3e54409fcb92656a80597a4
* Dashboard with existing defrag panels:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone%20-%20OM%20Snapshot.json#L2600
* SST pruning metric definitions:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/SSTFilePruningMetrics.java#L34
Related dashboard work: HDDS-13181.
cc [~smeng]
* Deletion metric definitions:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DeletingServiceMetrics.java#L109
was:
h2. Remaining scope: SST pruning and snapshot deletion metrics in Grafana
The Ozone - OM Snapshot.json Grafana dashboard already contains Snapshot Defrag
Metrics panels, including full/incremental defrag counts, failures, skipped
snapshots, tables compacted, and delta files processed. Do not recreate those
panels in this issue.
Add the remaining observability panels to the existing dashboard:
# SSTFilePruningMetrics: filesPrunedTotal, filesPrunedLast, filesSkippedTotal,
compactionsProcessed, pruneQueueSize, and pruningFailures.
# DeletingServiceMetrics snapshot gauges: snapKeysReclaimedLast,
snapReclaimedSizeLast, snapKeysIteratedLast, and snapKeysNotReclaimableLast.
Use the actual exported Prometheus metric names and labels from these metric
sources. Include appropriate units, descriptions, and OM-instance filtering,
following existing dashboard conventions. Verify the panels against exported
metrics so queries refer to real series.
h2. Existing implementation and metric sources
Repository baseline: d0949f291791c399f3e54409fcb92656a80597a4
* Dashboard with existing defrag panels:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone%20-%20OM%20Snapshot.json#L2600
* SST pruning metric definitions:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/SSTFilePruningMetrics.java#L34
Related dashboard work: HDDS-13181.
cc [~smeng]
* Deletion metric definitions:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DeletingServiceMetrics.java#L109
Summary: [Snapshot] Visualize SST pruning and snapshot deletion metrics
in Grafana (was: [Snapshot] Visualize Snapshot defragmentation metrics in
Grafana)
> [Snapshot] Visualize SST pruning and snapshot deletion metrics in Grafana
> -------------------------------------------------------------------------
>
> Key: HDDS-14993
> URL: https://issues.apache.org/jira/browse/HDDS-14993
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> h2. Completed scope
> HDDS-15256, implemented by [PR
> #10253|https://github.com/apache/ozone/pull/10253] and merged on 18 May 2026,
> added the Snapshot Defrag Metrics panels originally requested here. It
> includes overall/full/incremental defrag counts and failures, skipped
> snapshots, tables compacted, and delta files processed. These panels do not
> need to be implemented again.
> The PR did not add SSTFilePruningMetrics or the four DeletingServiceMetrics
> snapshot gauges listed below; those remain the scope of this issue.
> h2. Remaining scope: SST pruning and snapshot deletion metrics in Grafana
> Add the remaining observability panels to the existing dashboard:
> # SSTFilePruningMetrics: filesPrunedTotal, filesPrunedLast,
> filesSkippedTotal, compactionsProcessed, pruneQueueSize, and pruningFailures.
> # DeletingServiceMetrics snapshot gauges: snapKeysReclaimedLast,
> snapReclaimedSizeLast, snapKeysIteratedLast, and snapKeysNotReclaimableLast.
> Use the actual exported Prometheus metric names and labels from these metric
> sources. Include appropriate units, descriptions, and OM-instance filtering,
> following existing dashboard conventions. Verify the panels against exported
> metrics so queries refer to real series.
> h2. Existing implementation and metric sources
> Repository baseline: d0949f291791c399f3e54409fcb92656a80597a4
> * Dashboard with existing defrag panels:
> https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone%20-%20OM%20Snapshot.json#L2600
> * SST pruning metric definitions:
> https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/SSTFilePruningMetrics.java#L34
> Related dashboard work: HDDS-13181.
> cc [~smeng]
> * Deletion metric definitions:
> https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DeletingServiceMetrics.java#L109
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
