codeant-ai-for-open-source[bot] commented on code in PR #40238: URL: https://github.com/apache/superset/pull/40238#discussion_r3780035136
########## docs/admin_docs/configuration/dashboard-performance.mdx: ########## @@ -0,0 +1,174 @@ +--- +title: Dashboard Performance +hide_title: true +sidebar_position: 5 +version: 1 +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Dashboard Performance + +A dashboard's perceived speed is determined by three independent things: how +many charts have to render, how many queries the backend can execute +concurrently, and how quickly the underlying data warehouse can return +results. Superset gives you levers for the first two; the third belongs to +your warehouse. This page covers the dashboard-side levers and the practical +guidance around them. + +## Is there a maximum chart count per dashboard? + +**No hard limit is enforced** — Superset has no configuration key that +caps the number of charts on a dashboard. In practice, dashboards behave +well up to a few dozen charts. Beyond that, you'll typically feel friction +on the initial load and during cross-filter / time-range updates, even with +the lazy-loading optimizations described below. + +Rough thresholds to keep in mind: + +- **Under ~25 charts**: usually no perceptible problem. +- **25–50 charts**: still fine, but you start to want tabs to break the + page into chunks the user actually looks at. +- **Over ~50 charts**: split into multiple dashboards or use tabs + aggressively. The bottleneck is rarely Superset itself — it's the + warehouse executing dozens of queries in parallel and the browser + rendering dozens of chart frames. + +These are guidelines, not guarantees. A dashboard of 100 sparkline-style +charts hitting a fast cache behaves very differently from a dashboard of +20 heavy aggregations against a cold warehouse. + +## Lazy rendering — `DASHBOARD_VIRTUALIZATION` + +Superset's dashboard layout is virtualized at the row level. Charts that +are far below the user's current scroll position render a placeholder +instead of their visualization until the user scrolls them into view, and +go back to a placeholder if scrolled well past. The chart component itself +stays mounted throughout — only the visualization is swapped for a +placeholder — so this alone does **not** reduce backend query load; see +[Deferred data fetch](#deferred-data-fetch--dashboard_virtualization_defer_data) +below for that. This is on by default. + +**Feature flag**: `DASHBOARD_VIRTUALIZATION` (default: `True`) + +The flag is `stable` and marked for path-to-deprecation — meaning the +behavior will eventually be non-optional, but the flag still exists so +operators can disable it if a specific layout misbehaves. + +**Behavior** (from `superset-frontend/src/dashboard/components/gridComponents/Row/Row.tsx`): + +- A chart's visualization is rendered when its row scrolls within **1 + viewport height** of the visible area. +- A chart's visualization is swapped back for a placeholder when its row + scrolls more than **4 viewport heights** away from the visible area. +- Tabs that aren't currently selected don't render their content at all + (see below). +- The placeholder-swap-back is skipped in **embedded** mode (so an + embedded dashboard keeps its charts rendered once they've been seen, + which avoids re-rendering on scroll-up). Both halves are skipped for + **headless / bot** rendering (so screenshot / report jobs load every + chart). + +## Deferred data fetch — `DASHBOARD_VIRTUALIZATION_DEFER_DATA` + +By default, `DASHBOARD_VIRTUALIZATION` only controls whether a chart's +*visualization* is rendered — the chart component still mounts and issues +its data request immediately, regardless of scroll position. +`DASHBOARD_VIRTUALIZATION_DEFER_DATA` is a supplementary flag that skips +the data request itself for charts that aren't currently in view, useful +for backends where opening a connection or compiling a query is expensive +even if the result would be thrown away. + +**Feature flag**: `DASHBOARD_VIRTUALIZATION_DEFER_DATA` (default: `False`) + +Enable this if you see warehouse load spike on dashboard *open* even +though most charts are off-screen. + +## Per-tab lazy loading + +**This is on by default and has no flag.** A tab's content is not rendered +until the user activates that tab, so charts inside an unselected tab do +not fetch data on dashboard open. When the user clicks the tab, that +tab's charts mount and fetch in the normal way. + +Practically: tabs are the single most effective tool for a large +dashboard. Splitting 60 charts across 4 tabs effectively turns dashboard +open into "load ~15 charts," and the remaining ones lazy-load only if the +user goes looking. + +## Is there a switch to cap concurrent chart queries? + +**No.** Superset does not implement a frontend-side concurrent-request +limiter. Each chart issues its own data request when it mounts, and the +browser handles parallelism (typically ~6 in-flight HTTP requests per +origin, then the rest queue). Backend throughput is bounded by your +Gunicorn worker count for synchronous query execution, or by your Celery +worker pool when [async queries](./async-queries-celery.mdx) are enabled. Review Comment: **Suggestion:** The stated six-request-per-origin queue is not a Superset-wide browser behavior. It mainly describes older HTTP/1.1 connection limits; modern browsers can multiplex many requests over HTTP/2 or HTTP/3, and the effective limit depends on the protocol and server configuration. Presenting this as the expected concurrency ceiling can lead operators to believe the browser is throttling warehouse load when it may not be. [possible bug] <details> <summary><b>Severity Level:</b> Minor 🧹</summary> ```mdx - ⚠️ Browser concurrency varies by deployment protocol. - ⚠️ Operators may underestimate frontend request concurrency. - ⚠️ Warehouse-load expectations can be inaccurate. ``` </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=c1764b37de3c40be9093dc664a126afc&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=c1764b37de3c40be9093dc664a126afc&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) <details> <summary><b>Prompt for AI Agent 🤖 </b></summary> ```mdx This is a comment left during a code review. **Path:** docs/admin_docs/configuration/dashboard-performance.mdx **Line:** 119:123 **Comment:** *Possible Bug: The stated six-request-per-origin queue is not a Superset-wide browser behavior. It mainly describes older HTTP/1.1 connection limits; modern browsers can multiplex many requests over HTTP/2 or HTTP/3, and the effective limit depends on the protocol and server configuration. Presenting this as the expected concurrency ceiling can lead operators to believe the browser is throttling warehouse load when it may not be. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40238&comment_hash=464863edad63cc9e43f7e9450f6d18a0f185817987118bcb89e7f737adb93cfd&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40238&comment_hash=464863edad63cc9e43f7e9450f6d18a0f185817987118bcb89e7f737adb93cfd&reaction=dislike'>👎</a> ########## docs/admin_docs/configuration/dashboard-performance.mdx: ########## @@ -0,0 +1,174 @@ +--- +title: Dashboard Performance +hide_title: true +sidebar_position: 5 +version: 1 +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Dashboard Performance + +A dashboard's perceived speed is determined by three independent things: how +many charts have to render, how many queries the backend can execute +concurrently, and how quickly the underlying data warehouse can return +results. Superset gives you levers for the first two; the third belongs to +your warehouse. This page covers the dashboard-side levers and the practical +guidance around them. + +## Is there a maximum chart count per dashboard? + +**No hard limit is enforced** — Superset has no configuration key that +caps the number of charts on a dashboard. In practice, dashboards behave +well up to a few dozen charts. Beyond that, you'll typically feel friction +on the initial load and during cross-filter / time-range updates, even with +the lazy-loading optimizations described below. + +Rough thresholds to keep in mind: + +- **Under ~25 charts**: usually no perceptible problem. +- **25–50 charts**: still fine, but you start to want tabs to break the + page into chunks the user actually looks at. +- **Over ~50 charts**: split into multiple dashboards or use tabs + aggressively. The bottleneck is rarely Superset itself — it's the + warehouse executing dozens of queries in parallel and the browser + rendering dozens of chart frames. + +These are guidelines, not guarantees. A dashboard of 100 sparkline-style +charts hitting a fast cache behaves very differently from a dashboard of +20 heavy aggregations against a cold warehouse. + +## Lazy rendering — `DASHBOARD_VIRTUALIZATION` + +Superset's dashboard layout is virtualized at the row level. Charts that +are far below the user's current scroll position render a placeholder +instead of their visualization until the user scrolls them into view, and +go back to a placeholder if scrolled well past. The chart component itself +stays mounted throughout — only the visualization is swapped for a +placeholder — so this alone does **not** reduce backend query load; see +[Deferred data fetch](#deferred-data-fetch--dashboard_virtualization_defer_data) +below for that. This is on by default. + +**Feature flag**: `DASHBOARD_VIRTUALIZATION` (default: `True`) + +The flag is `stable` and marked for path-to-deprecation — meaning the +behavior will eventually be non-optional, but the flag still exists so +operators can disable it if a specific layout misbehaves. + +**Behavior** (from `superset-frontend/src/dashboard/components/gridComponents/Row/Row.tsx`): + +- A chart's visualization is rendered when its row scrolls within **1 + viewport height** of the visible area. +- A chart's visualization is swapped back for a placeholder when its row + scrolls more than **4 viewport heights** away from the visible area. +- Tabs that aren't currently selected don't render their content at all + (see below). +- The placeholder-swap-back is skipped in **embedded** mode (so an + embedded dashboard keeps its charts rendered once they've been seen, + which avoids re-rendering on scroll-up). Both halves are skipped for + **headless / bot** rendering (so screenshot / report jobs load every + chart). + +## Deferred data fetch — `DASHBOARD_VIRTUALIZATION_DEFER_DATA` + +By default, `DASHBOARD_VIRTUALIZATION` only controls whether a chart's +*visualization* is rendered — the chart component still mounts and issues +its data request immediately, regardless of scroll position. +`DASHBOARD_VIRTUALIZATION_DEFER_DATA` is a supplementary flag that skips +the data request itself for charts that aren't currently in view, useful +for backends where opening a connection or compiling a query is expensive +even if the result would be thrown away. Review Comment: **Suggestion:** `DASHBOARD_VIRTUALIZATION_DEFER_DATA` is not an independent data-fetch limiter: `Chart.runQuery` skips the request only when `shouldRenderChart()` is false, and that method returns true whenever `DASHBOARD_VIRTUALIZATION` is disabled. Operators who disable virtualization and enable only the defer flag will see no reduction in dashboard-open queries, so document the dependency explicitly. [api mismatch] <details> <summary><b>Severity Level:</b> Major ⚠️</summary> ```mdx - ⚠️ Disabled virtualization defeats deferred dashboard queries. - ❌ Operators may still overload warehouses during dashboard open. - ⚠️ Configuration guidance can produce unexpected query volume. ``` </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=b229741a30ee4ec592d30912dfe1cb47&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=b229741a30ee4ec592d30912dfe1cb47&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) <details> <summary><b>Prompt for AI Agent 🤖 </b></summary> ```mdx This is a comment left during a code review. **Path:** docs/admin_docs/configuration/dashboard-performance.mdx **Line:** 94:97 **Comment:** *Api Mismatch: `DASHBOARD_VIRTUALIZATION_DEFER_DATA` is not an independent data-fetch limiter: `Chart.runQuery` skips the request only when `shouldRenderChart()` is false, and that method returns true whenever `DASHBOARD_VIRTUALIZATION` is disabled. Operators who disable virtualization and enable only the defer flag will see no reduction in dashboard-open queries, so document the dependency explicitly. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40238&comment_hash=6c6f0f21775a06f8fad6c972bdb5766a19cb503bd9b8afa2a437970bbdd0e385&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40238&comment_hash=6c6f0f21775a06f8fad6c972bdb5766a19cb503bd9b8afa2a437970bbdd0e385&reaction=dislike'>👎</a> ########## docs/admin_docs/configuration/dashboard-performance.mdx: ########## @@ -0,0 +1,174 @@ +--- +title: Dashboard Performance +hide_title: true +sidebar_position: 5 +version: 1 +--- + +<!-- +Licensed to the Apache Software Foundation (ASF) under one +or more contributor license agreements. See the NOTICE file +distributed with this work for additional information +regarding copyright ownership. The ASF licenses this file +to you under the Apache License, Version 2.0 (the +"License"); you may not use this file except in compliance +with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, +software distributed under the License is distributed on an +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +KIND, either express or implied. See the License for the +specific language governing permissions and limitations +under the License. +--> + +# Dashboard Performance + +A dashboard's perceived speed is determined by three independent things: how +many charts have to render, how many queries the backend can execute +concurrently, and how quickly the underlying data warehouse can return +results. Superset gives you levers for the first two; the third belongs to +your warehouse. This page covers the dashboard-side levers and the practical +guidance around them. + +## Is there a maximum chart count per dashboard? + +**No hard limit is enforced** — Superset has no configuration key that +caps the number of charts on a dashboard. In practice, dashboards behave +well up to a few dozen charts. Beyond that, you'll typically feel friction +on the initial load and during cross-filter / time-range updates, even with +the lazy-loading optimizations described below. + +Rough thresholds to keep in mind: + +- **Under ~25 charts**: usually no perceptible problem. +- **25–50 charts**: still fine, but you start to want tabs to break the + page into chunks the user actually looks at. +- **Over ~50 charts**: split into multiple dashboards or use tabs + aggressively. The bottleneck is rarely Superset itself — it's the + warehouse executing dozens of queries in parallel and the browser + rendering dozens of chart frames. + +These are guidelines, not guarantees. A dashboard of 100 sparkline-style +charts hitting a fast cache behaves very differently from a dashboard of +20 heavy aggregations against a cold warehouse. + +## Lazy rendering — `DASHBOARD_VIRTUALIZATION` + +Superset's dashboard layout is virtualized at the row level. Charts that +are far below the user's current scroll position render a placeholder +instead of their visualization until the user scrolls them into view, and +go back to a placeholder if scrolled well past. The chart component itself +stays mounted throughout — only the visualization is swapped for a +placeholder — so this alone does **not** reduce backend query load; see +[Deferred data fetch](#deferred-data-fetch--dashboard_virtualization_defer_data) +below for that. This is on by default. + +**Feature flag**: `DASHBOARD_VIRTUALIZATION` (default: `True`) + +The flag is `stable` and marked for path-to-deprecation — meaning the +behavior will eventually be non-optional, but the flag still exists so +operators can disable it if a specific layout misbehaves. + +**Behavior** (from `superset-frontend/src/dashboard/components/gridComponents/Row/Row.tsx`): + +- A chart's visualization is rendered when its row scrolls within **1 + viewport height** of the visible area. +- A chart's visualization is swapped back for a placeholder when its row + scrolls more than **4 viewport heights** away from the visible area. +- Tabs that aren't currently selected don't render their content at all + (see below). +- The placeholder-swap-back is skipped in **embedded** mode (so an + embedded dashboard keeps its charts rendered once they've been seen, + which avoids re-rendering on scroll-up). Both halves are skipped for + **headless / bot** rendering (so screenshot / report jobs load every + chart). + +## Deferred data fetch — `DASHBOARD_VIRTUALIZATION_DEFER_DATA` + +By default, `DASHBOARD_VIRTUALIZATION` only controls whether a chart's +*visualization* is rendered — the chart component still mounts and issues +its data request immediately, regardless of scroll position. +`DASHBOARD_VIRTUALIZATION_DEFER_DATA` is a supplementary flag that skips +the data request itself for charts that aren't currently in view, useful +for backends where opening a connection or compiling a query is expensive +even if the result would be thrown away. + +**Feature flag**: `DASHBOARD_VIRTUALIZATION_DEFER_DATA` (default: `False`) + +Enable this if you see warehouse load spike on dashboard *open* even +though most charts are off-screen. + +## Per-tab lazy loading + +**This is on by default and has no flag.** A tab's content is not rendered +until the user activates that tab, so charts inside an unselected tab do +not fetch data on dashboard open. When the user clicks the tab, that +tab's charts mount and fetch in the normal way. + +Practically: tabs are the single most effective tool for a large +dashboard. Splitting 60 charts across 4 tabs effectively turns dashboard +open into "load ~15 charts," and the remaining ones lazy-load only if the +user goes looking. + +## Is there a switch to cap concurrent chart queries? + +**No.** Superset does not implement a frontend-side concurrent-request +limiter. Each chart issues its own data request when it mounts, and the +browser handles parallelism (typically ~6 in-flight HTTP requests per +origin, then the rest queue). Backend throughput is bounded by your +Gunicorn worker count for synchronous query execution, or by your Celery +worker pool when [async queries](./async-queries-celery.mdx) are enabled. + +If you need to throttle warehouse load, the right place is: + +1. The warehouse itself (connection pool / concurrency limits). +2. Superset's Celery configuration (smaller worker pool when async + queries are on). +3. Splitting heavy charts across tabs or separate dashboards (each + dashboard load only fetches what's visible). + +## Splitting strategies + +When a dashboard outgrows comfortable performance, the options in order +of effort: + +**1. Move sections into tabs.** Same dashboard, but only the active tab's +charts fetch. This is the cheapest change and often the only one needed. + +**2. Cache aggressively.** A Redis cache backend (see +[Caching](./cache.mdx)) means repeat dashboard loads serve from cache +rather than re-hitting the warehouse. This is especially impactful for +dashboards opened by many users in close succession. + +**3. Enable async queries.** [Async query execution](./async-queries-celery.mdx) +via Celery decouples query duration from request lifetime, so a slow +chart doesn't block the page. The user sees other charts come in as +their queries complete. + +**4. Split into multiple dashboards.** Group related charts into purpose- +specific dashboards rather than one mega-dashboard. Link them from a +landing dashboard or a navigation menu. + +**5. Pre-aggregate at the warehouse level.** If the same expensive +aggregation appears across many charts, materialize it as a view or +scheduled table in the warehouse so each chart query is a cheap lookup. + +## Operational notes + +- The feature flags above are set in `superset_config.py`, e.g.: + + ```python + FEATURE_FLAGS = { + "DASHBOARD_VIRTUALIZATION": True, + "DASHBOARD_VIRTUALIZATION_DEFER_DATA": True, + } + ``` + +- See [Feature Flags](./feature-flags.mdx) for the full list of supported + flags and their lifecycle stages. +- Report and screenshot jobs (alerts, scheduled reports, dashboard + exports) intentionally bypass row virtualization so the rendered + artifact includes every chart, not just the ones above the fold. Review Comment: **Suggestion:** The claim that all listed report, alert, scheduled-report, and dashboard-export paths bypass row virtualization is broader than the frontend contract. The bypass is controlled by `navigator.webdriver` through `isCurrentUserBot()`, while dashboard metadata exports do not render the dashboard in a webdriver-controlled page at all. This can cause operators to expect every export artifact to contain fully rendered charts when the export path does not use this rendering behavior. [api mismatch] <details> <summary><b>Severity Level:</b> Minor 🧹</summary> ```mdx - ⚠️ Metadata exports do not render dashboard charts. - ⚠️ Operators may misinterpret export rendering behavior. - ⚠️ Export troubleshooting guidance becomes ambiguous. ``` </details> [](https://app.codeant.ai/fix-in-ide?tool=cursor&prompt_id=633a8747ee7b4c2aae6948915df50278&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) [](https://app.codeant.ai/fix-in-ide?tool=vscode-claude&prompt_id=633a8747ee7b4c2aae6948915df50278&service=github&base_url=https%3A%2F%2Fgithub.com&org=apache&repo=apache%2Fsuperset) <details> <summary><b>Prompt for AI Agent 🤖 </b></summary> ```mdx This is a comment left during a code review. **Path:** docs/admin_docs/configuration/dashboard-performance.mdx **Line:** 172:174 **Comment:** *Api Mismatch: The claim that all listed report, alert, scheduled-report, and dashboard-export paths bypass row virtualization is broader than the frontend contract. The bypass is controlled by `navigator.webdriver` through `isCurrentUserBot()`, while dashboard metadata exports do not render the dashboard in a webdriver-controlled page at all. This can cause operators to expect every export artifact to contain fully rendered charts when the export path does not use this rendering behavior. Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise. Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix ``` </details> <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40238&comment_hash=a806e3a469faafb8f03bf04ad5e046b4e6f50ca4d00a2cffeacc33090742a7e3&reaction=like'>👍</a> | <a href='https://app.codeant.ai/feedback?pr_url=https%3A%2F%2Fgithub.com%2Fapache%2Fsuperset%2Fpull%2F40238&comment_hash=a806e3a469faafb8f03bf04ad5e046b4e6f50ca4d00a2cffeacc33090742a7e3&reaction=dislike'>👎</a> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
