This is an automated email from the ASF dual-hosted git repository. ArafatKhan2198 pushed a commit to branch jira4 in repository https://gitbox.apache.org/repos/asf/ozone-site.git
commit 7fdb0b935f9760468951293c4e13e42cb5ec6a4d Author: arafat <[email protected]> AuthorDate: Sat Jul 25 15:13:37 2026 +0530 HDDS-XXXX. Add Recon Web UI analytics pages: Insights, OM DB Insights - Add 08-insights.mdx and 09-om-db-insights.mdx with full 9-section format - Move screenshots into the web-ui folder - Replace Insights screenshot with updated version - Add AI Assistant cross-link and remove Heatmap from README.mdx Co-Authored-By: Claude <[email protected]> --- .../02-recon/01-recon-web-ui/08-insights.mdx | 118 +++++++++++++++ .../02-recon/01-recon-web-ui/09-om-db-insights.mdx | 166 +++++++++++++++++++++ .../02-recon/01-recon-web-ui/README.mdx | 8 +- .../01-recon-web-ui/ozone_recon_insights.png | Bin 0 -> 294284 bytes .../ozone_recon_omdb_insights.png | Bin .../02-recon/ozone_recon_insights.png | Bin 253402 -> 0 bytes 6 files changed, 291 insertions(+), 1 deletion(-) diff --git a/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/08-insights.mdx b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/08-insights.mdx new file mode 100644 index 0000000000..9185d06f4a --- /dev/null +++ b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/08-insights.mdx @@ -0,0 +1,118 @@ +--- +sidebar_label: Insights +--- + +# Recon UI — Insights Page + + + +## 1. Page Overview + +The **Insights** page visualizes how data is distributed across the cluster by +size. It shows two charts: a **File Size Frequency Distribution** (how many +files fall into each size range) and a **Container Size Distribution** (how +containers are spread across size ranges). The file chart can be narrowed to +specific volumes and buckets. + +It is a read-only analytics page for understanding data-size patterns — for +example, whether the cluster is dominated by many small files or a few large +ones. + +> Note: this is the **Insights** page. A separate **OM DB Insights** page +> covers open keys, keys pending deletion, and related OM database details. + +## 2. When to Use This Page + +- To understand the file-size profile of the cluster or of a specific bucket. +- To detect a "small files problem" (a very high count of tiny files). +- To see how containers are distributed by size. +- To compare data-size patterns between volumes or buckets. + +## 3. How to Access the Page + +Open the **Insights** entry in the left navigation menu, or go to the +`/Insights` route directly. + +## 4. Information Displayed + +The page header shows the title. While data loads, a **"Charts are being +loaded"** message is shown. Once loaded, two cards appear side by side. + +### File Size Frequency Distribution (bar chart) + +- A bar chart where the **x-axis** is the file-size range (for example + `512 KiB - 1 MiB`) and the **y-axis** is the **Number of Files** in that + range. +- Ranges are power-of-two size buckets; hovering a bar shows the exact size + range and count. +- Two filters above the chart: + - **Volumes** — multi-select (with search and select-all) to include specific + volumes. + - **Buckets** — multi-select, enabled only when exactly one volume is + selected; it lists that volume's buckets. + +### Container Size Distribution (pie chart) + +- A pie chart of containers grouped into size ranges; each slice is a range and + its value is the number of containers in that range. +- Hovering a slice shows the size range and container count. +- This chart has no volume/bucket filter (containers are not tied to a single + bucket). + +## 5. Available Actions + +- **Volumes filter** (file chart) — choose which volumes to include; supports + search and select-all. +- **Buckets filter** (file chart) — choose buckets within a single selected + volume. It is disabled when more than one volume is selected (unless the + cluster has only one volume). +- **Chart tooltips** — hover any bar or slice for the exact size range and count. + +There is no auto-refresh panel, search box, table, or download on this page. + +## 6. How to Interpret the Information + +- **Tall bars at small size ranges:** many small files. A very large count of + tiny files can hurt performance and metadata scaling (the "small files + problem"). +- **Bars concentrated at large size ranges:** the cluster is dominated by large + files. +- **Container pie skewed to small ranges:** many small or partially filled + containers; skewed to large ranges means containers are well filled. +- **Filtering the file chart:** select a single volume, then its buckets, to see + the size profile for just that bucket — useful for comparing workloads. +- **"No Data" result:** the corresponding metric has no data yet (for example a + new cluster with no files). This is expected, not an error. +- **Error banner inside a chart** ("No data available. …"): the underlying data + could not be fetched. + +## 7. Common Use Cases + +1. **Diagnose a small-files problem.** Open the file chart for the whole cluster + and check whether the smallest size ranges dominate the file count. +2. **Profile one bucket.** Select a single volume and one of its buckets to see + that bucket's file-size distribution in isolation. +3. **Assess container fill.** Use the container pie chart to judge whether + containers are mostly small (poorly filled) or large (well filled). + +## 8. Important Notes and Limitations + +- **Data source and freshness.** Both charts are built from counts Recon + maintains in its own database (file-count-by-size and container-count-by-size), + populated by Recon background tasks that process its copy of the Ozone Manager + (OM) data and container information. The charts reflect the last time those + tasks ran, not real time. +- **Bucket filter needs a single volume.** The Buckets filter is only active when + exactly one volume is selected; with multiple volumes it is disabled. +- **Size ranges are power-of-two buckets**, so exact per-file sizes are not + shown — only how many files fall into each range. +- The page has **no manual refresh control**; reload the page to re-fetch. +- On no data, each chart shows a "No Data" message independently. + +## 9. Related Pages + +- **OM DB Insights** — open keys, keys pending deletion, and OM database + details. +- **Namespace Usage** — size usage explored by path rather than by size range. +- **Containers** — container health and per-state detail. +- **Overview** — cluster-wide totals. diff --git a/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/09-om-db-insights.mdx b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/09-om-db-insights.mdx new file mode 100644 index 0000000000..588d34250a --- /dev/null +++ b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/09-om-db-insights.mdx @@ -0,0 +1,166 @@ +--- +sidebar_label: OM DB Insights +--- + +# Recon UI — OM DB Insights Page + + + +## 1. Page Overview + +The **OM DB Insights** page exposes detail from the Ozone Manager (OM) database +and how it lines up with SCM's container view. It is organized into five tabs: +container mismatches between OM and SCM, open (uncommitted) keys, keys pending +deletion, keys mapped to deleted containers, and directories pending deletion. + +It is the drill-down destination for the Open Keys and Delete Pending Keys +summaries on the Overview page, and the main place to investigate deletion +backlogs and OM/SCM inconsistencies. + +> Note: this is the **OM DB Insights** page. The separately named **Insights** +> page covers file-size and container-size charts. + +## 2. When to Use This Page + +- To investigate a growing backlog of keys or directories pending deletion. +- To find open keys that were never committed (possible leaked/abandoned writes). +- To find containers that exist in OM but not SCM (or vice versa). +- To see which keys map to containers SCM has marked deleted. +- To list the keys inside a specific container. + +## 3. How to Access the Page + +Open the **OM DB Insights** entry in the left navigation menu, or go to the +`/Om` route directly. From the **Overview** page, the **Open Keys Summary** card +opens this page on the Open Keys tab, and the **Delete Pending Keys Summary** +card opens it on the Keys Pending for Deletion tab. + +## 4. Information Displayed + +The page header shows the title. Below it are five tabs. Each tab has a **Limit** +selector and a table; some tables allow row expansion to list the affected keys. + +### Tab 1 — Container Mismatch Info + +Containers present in OM but missing in SCM, or the reverse. Columns: + +- **Container ID** +- **Count Of Keys** — number of keys in the container. +- **Pipelines** — the pipelines involved. +- **Exists at** — where the container exists (OM or SCM). + +A toggle switches between containers **missing in OM** and **missing in SCM**. +Rows expand to show the keys in the container. + +### Tab 2 — Open Keys + +Keys that are being written but not yet committed. Columns: + +- **Key Name** +- **Size** +- **Path** +- **In state since** — how long the key has been open. +- **Replication Type** and **Replication Factor** +- **Type** — FSO or Non-FSO. + +A toggle switches between **FSO** and **Non-FSO** keys. + +### Tab 3 — Keys Pending for Deletion + +Keys awaiting deletion. Columns: + +- **Key Name** +- **Path** +- **Total Data Size** +- **Total Key Count** + +Rows expand to show the individual pending-delete key entries. + +### Tab 4 — Deleted Container Keys + +Keys mapped to containers that SCM has marked as **DELETED**. Columns: + +- **Container ID** +- **Count Of Keys** +- **Pipelines** + +Rows expand to show the keys in the container. + +### Tab 5 — Directories Pending for Deletion + +Directories awaiting deletion. Columns: + +- **Directory Name** +- **In state since** +- **Path** +- **Size** + +### Expanded key table (shared) + +When a container row is expanded, the keys are listed with **Volume**, +**Bucket**, **Key**, **Size**, **Date Created**, and **Date Modified**. + +## 5. Available Actions + +- **Tabs** — switch between the five views. +- **Limit** selector — how many rows to fetch (1000, 5000, 10000, 20000; default + 1000). The selection is shared across tabs. +- **Toggles:** + - Container Mismatch: **missing in OM** vs **missing in SCM**. + - Open Keys: **FSO** vs **Non-FSO**. +- **Search** — filter the loaded rows within a tab. +- **Row expand** — on the container-based tabs, expand a row to load and view + its keys. +- **Pagination** — page through rows; the footer shows the range and total. + +There is no auto-refresh panel on this page. + +## 6. How to Interpret the Information + +- **Container Mismatch entries:** a healthy cluster generally has none. Entries + can appear transiently while OM and SCM converge; persistent mismatches warrant + investigation. The **Exists at** column tells you which service still has the + container. +- **Open Keys that are old (large "In state since"):** may be abandoned or + leaked writes holding space; a large open-keys total also shows on the Overview + Open Keys summary. +- **Keys / Directories Pending for Deletion growing over time:** the background + deletion services may be falling behind; a steadily rising backlog is the + signal to investigate. +- **Deleted Container Keys:** keys still referencing containers SCM considers + deleted — useful when tracing why space is not being reclaimed. +- **FSO vs Non-FSO:** file-system-optimized buckets vs object-store/legacy + buckets; the toggle lets you look at each layout's open keys separately. + +## 7. Common Use Cases + +1. **Diagnose a deletion backlog.** From the Overview Delete Pending Keys card, + land on the Keys Pending for Deletion tab, raise the Limit, and review total + size and counts to gauge the backlog. +2. **Find leaked open keys.** On the Open Keys tab, sort/scan by "In state since" + to find keys that have been open a long time and were likely never committed. +3. **Investigate OM/SCM drift.** On the Container Mismatch tab, toggle between + missing-in-OM and missing-in-SCM and expand rows to see exactly which keys are + affected. + +## 8. Important Notes and Limitations + +- **Data source and freshness.** All tabs read from Recon's own copy of the OM + database (and its comparison against SCM's container view for the mismatch + tabs), updated by periodic sync. Values reflect the last sync, not real time, + and there is no auto-refresh control on this page — reload to re-fetch. +- **Limit caps the result set.** Only up to the selected limit of rows is + fetched per tab; raise the limit on large clusters, keeping in mind larger + limits take longer to load. +- **Search applies to the loaded rows only.** +- **Mismatches can be transient** during normal OM/SCM synchronization and do not + always indicate a problem. +- On an error, the page shows a data-fetch error and the affected table stays + empty. + +## 9. Related Pages + +- **Overview** — the Open Keys and Delete Pending Keys summary cards link here. +- **Containers** — container health and unhealthy-state detail. +- **Insights** — file-size and container-size distribution charts. +- **Namespace Usage** — size usage explored by path. diff --git a/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/README.mdx b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/README.mdx index cb0a8d896d..dd3c7f9aef 100644 --- a/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/README.mdx +++ b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/README.mdx @@ -29,7 +29,6 @@ The left navigation panel provides access to all available views: - Insights - OM DB Insights - Namespace Usage -- Heatmap The **Cluster Capacity** page is reached from the **View More** link on the Overview page rather than from the left navigation. @@ -43,4 +42,11 @@ Most Recon UI pages share the same controls and behaviors. They are described on - **Limit selector** — on large listings, controls how many rows are fetched from the server (options such as 1000/5000/10000/20000; default 1000). **Search, sorting, and filters apply to the loaded rows only**, not to rows beyond the limit — raise the limit if you expect more. - **Error states** — on a fetch error, a page reports a data-fetch error and its table stays empty; until the first successful load, counts may appear as zero or `N/A`. +## AI Assistant + +Recon includes an optional AI Assistant that lets you query cluster metadata in +plain English from within the UI. It is disabled by default and requires +operator configuration. See the [Recon AI Assistant](../recon-ai-assistant) page +for setup, configuration, security, and usage guidance. + <DocCardList/> diff --git a/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/ozone_recon_insights.png b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/ozone_recon_insights.png new file mode 100644 index 0000000000..232141e9de Binary files /dev/null and b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/ozone_recon_insights.png differ diff --git a/docs/05-administrator-guide/03-operations/09-observability/02-recon/ozone_recon_omdb_insights.png b/docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/ozone_recon_omdb_insights.png similarity index 100% rename from docs/05-administrator-guide/03-operations/09-observability/02-recon/ozone_recon_omdb_insights.png rename to docs/05-administrator-guide/03-operations/09-observability/02-recon/01-recon-web-ui/ozone_recon_omdb_insights.png diff --git a/docs/05-administrator-guide/03-operations/09-observability/02-recon/ozone_recon_insights.png b/docs/05-administrator-guide/03-operations/09-observability/02-recon/ozone_recon_insights.png deleted file mode 100644 index e19d89554f..0000000000 Binary files a/docs/05-administrator-guide/03-operations/09-observability/02-recon/ozone_recon_insights.png and /dev/null differ --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
