[
https://issues.apache.org/jira/browse/HDDS-14993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei-Chiu Chuang updated HDDS-14993:
-----------------------------------
Description:
h2. Remaining scope: SST pruning and snapshot deletion metrics in Grafana
The Ozone - OM Snapshot.json Grafana dashboard already contains Snapshot Defrag
Metrics panels, including full/incremental defrag counts, failures, skipped
snapshots, tables compacted, and delta files processed. Do not recreate those
panels in this issue.
Add the remaining observability panels to the existing dashboard:
# SSTFilePruningMetrics: filesPrunedTotal, filesPrunedLast, filesSkippedTotal,
compactionsProcessed, pruneQueueSize, and pruningFailures.
# DeletingServiceMetrics snapshot gauges: snapKeysReclaimedLast,
snapReclaimedSizeLast, snapKeysIteratedLast, and snapKeysNotReclaimableLast.
Use the actual exported Prometheus metric names and labels from these metric
sources. Include appropriate units, descriptions, and OM-instance filtering,
following existing dashboard conventions. Verify the panels against exported
metrics so queries refer to real series.
h2. Existing implementation and metric sources
Repository baseline: d0949f291791c399f3e54409fcb92656a80597a4
* Dashboard with existing defrag panels:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone%20-%20OM%20Snapshot.json#L2600
* SST pruning metric definitions:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/SSTFilePruningMetrics.java#L34
Related dashboard work: HDDS-13181.
cc [~smeng]
* Deletion metric definitions:
https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DeletingServiceMetrics.java#L109
was:
Similar to HDDS-13181
let's add snapshot defragmentation metrics into Ozone - OM Snapshot.json
dashboard definition.
Include SSTFilePruningMetrics too.
and:
snapKeysReclaimedLast, snapReclaimedSizeLast, snapKeysIteratedLast,
snapKeysNotReclaimableLast from DeletingServiceMetrics
cc [~smeng]
> [Snapshot] Visualize Snapshot defragmentation metrics in Grafana
> ----------------------------------------------------------------
>
> Key: HDDS-14993
> URL: https://issues.apache.org/jira/browse/HDDS-14993
> Project: Apache Ozone
> Issue Type: Task
> Reporter: Wei-Chiu Chuang
> Priority: Major
>
> h2. Remaining scope: SST pruning and snapshot deletion metrics in Grafana
> The Ozone - OM Snapshot.json Grafana dashboard already contains Snapshot
> Defrag Metrics panels, including full/incremental defrag counts, failures,
> skipped snapshots, tables compacted, and delta files processed. Do not
> recreate those panels in this issue.
> Add the remaining observability panels to the existing dashboard:
> # SSTFilePruningMetrics: filesPrunedTotal, filesPrunedLast,
> filesSkippedTotal, compactionsProcessed, pruneQueueSize, and pruningFailures.
> # DeletingServiceMetrics snapshot gauges: snapKeysReclaimedLast,
> snapReclaimedSizeLast, snapKeysIteratedLast, and snapKeysNotReclaimableLast.
> Use the actual exported Prometheus metric names and labels from these metric
> sources. Include appropriate units, descriptions, and OM-instance filtering,
> following existing dashboard conventions. Verify the panels against exported
> metrics so queries refer to real series.
> h2. Existing implementation and metric sources
> Repository baseline: d0949f291791c399f3e54409fcb92656a80597a4
> * Dashboard with existing defrag panels:
> https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/dist/src/main/compose/common/grafana/dashboards/Ozone%20-%20OM%20Snapshot.json#L2600
> * SST pruning metric definitions:
> https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-hdds/rocksdb-checkpoint-differ/src/main/java/org/apache/ozone/rocksdiff/SSTFilePruningMetrics.java#L34
> Related dashboard work: HDDS-13181.
> cc [~smeng]
> * Deletion metric definitions:
> https://github.com/apache/ozone/blob/d0949f291791c399f3e54409fcb92656a80597a4/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DeletingServiceMetrics.java#L109
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
