dosubot[bot] commented on issue #37091: URL: https://github.com/apache/superset/issues/37091#issuecomment-3743023094
<!-- Greeting --> Hi @beatricekay! I'm [Dosu](https://go.dosu.dev/dosubot) and Iām helping the superset team. <!-- Answer --> Yes, loading large datasets (like 30k points) in Superset Pivot Table charts causes performance issues and can result in missing or redacted data. The Pivot Table chart currently renders the entire dataset in the browser without pagination or virtualization, which often leads to browser hangs or incomplete rendering when the dataset is large. Filtering helps because it reduces the amount of data being rendered, making the chart more responsive and able to display more data, but it doesn't solve the underlying limitation. There is no pagination or virtualization support for Pivot Table charts in Superset as of versions 4.1.2 and 5.0.0, and there are no active plans or roadmap items to add this feature. This is a known limitation and has been discussed in issues and threads, with suggestions to use virtualization libraries like react-window, but nothing has been implemented yet ([source](https://github.com/apache/superset/issues/35981)). Exporting data from Pivot Table charts will export the entire dataset that is currently loaded (subject to the SQL_MAX_ROW config), not just a single page, since there is no pagination. If the dataset is too large, the export may be truncated at the configured row limit ([source](https://github.com/apache/superset/pull/33214)). Workarounds: use filters to reduce the dataset size, switch to Table charts (which support pagination and more robust export options), or use aggregate query modes to minimize the number of rows. If you have legacy pivot tables with performance issues, sometimes opening the chart in "Edit" mode and saving/overwriting it can help ([source](https://github.com/apache/superset/issues/31799)). For tracking progress or contributing to feature requests, see the related GitHub issues and discussions, especially [this one](https://github.com/apache/superset/issues/35981) on virtualization and performance. <!-- Dosu Comment Footer --> *To reply, just mention [@dosu](https://go.dosu.dev/dosubot).* --- <sup>How did I do? [Good](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=great_response) | [Irrelevant](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=irrelevant_answer) | [Incorrect](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=incorrect_sources) | [Verbose](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=too_verbose) | [Hallucination](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=hallucination) | [Report š](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=bug_report) | [Other](https://app.dosu.dev/response-feedback/4225d6e8-1ca1-41f7-baa4-394d04461a14?feedback_type=other)</sup> [](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=Yes%2C%20loading%20large%20datasets%20%28like%2030k%20points%29%20in%20Superset%20Pivot%20Table%20charts%20causes%20performance%20issues%20and%20can%20result%20in%20missing%20or%20redacted%20data.%20The%20Pivot%20Table%20chart%20currently%20renders%20the%20entire%20dataset%20in%20the%20browser%20without%20pagination%20or%20virtualization%2C%20which%20often%20leads%20to%20browser%20hangs%20or%20incomplete%20rendering%20when%20the%20dataset%20is%20large.%20Filtering%20helps%20because%20it%20reduces%20the%20amount%20of%20data%20being%20rendered%2C%20making%20the%20chart%20more%20responsive%20and%20able%20to%20display%20more%20data%2C%20but%20it%20doesn%27t%20solve%20the%20underlying%20limitation.%0A%0AThere%20is%20no%20pagination%20or%20virtualization %20support%20for%20Pivot%20Table%20charts%20in%20Superset%20as%20of%20versions%204.1.2%20and%205.0.0%2C%20and%20there%20are%20no%20active%20plans%20or%20roadmap%20items%20to%20add%20this%20feature.%20This%20is%20a%20known%20limitation%20and%20has%20been%20discussed%20in%20issues%20and%20threads%2C%20with%20suggestions%20to%20use%20virtualization%20libraries%20like%20react-window%2C%20but%20nothing%20has%20been%20implemented%20yet%20%28%5Bsource%5D%28https%3A//github.com/apache/superset/issues/35981%29%29.%0A%0AExporting%20data%20from%20Pivot%20Table%20charts%20will%20export%20the%20entire%20dataset%20that%20is%20currently%20loaded%20%28subject%20to%20the%20SQL_MAX_ROW%20config%29%2C%20not%20just%20a%20single%20page%2C%20since%20there%20is%20no%20pagination.%20If%20the%20dataset%20is%20too%20large%2C%20the%20export%20may%20be%20truncated%20at%20the%20configured%20row%20limit%20%28%5Bsource%5D%28https%3A//github.com/apache/superset/pull/33214%29%29.%0A%0AWorkarounds%3A%20use%20filters %20to%20reduce%20the%20dataset%20size%2C%20switch%20to%20Table%20charts%20%28which%20support%20pagination%20and%20more%20robust%20export%20options%29%2C%20or%20use%20aggregate%20query%20modes%20to%20minimize%20the%20number%20of%20rows.%20If%20you%20have%20legacy%20pivot%20tables%20with%20performance%20issues%2C%20sometimes%20opening%20the%20chart%20in%20%22Edit%22%20mode%20and%20saving/overwriting%20it%20can%20help%20%28%5Bsource%5D%28https%3A//github.com/apache/superset/issues/31799%29%29.%0A%0AFor%20tracking%20progress%20or%20contributing%20to%20feature%20requests%2C%20see%20the%20related%20GitHub%20issues%20and%20discussions%2C%20especially%20%5Bthis%20one%5D%28https%3A//github.com/apache/superset/issues/35981%29%20on%20virtualization%20and%20performance.) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/t weet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/37091) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
