rusackas opened a new issue, #44625: URL: https://github.com/apache/superset/issues/44625
### Bug description SIP-216 (#41184) fixed pivot table subtotals/totals being computed by re-aggregating already-aggregated cell values client-side, which is wrong for non-additive metrics (ratios, `COUNT_DISTINCT`, percentiles, etc.) — the live browser view now issues one query carrying `GROUPING SETS` so cells and totals come from the same real database-computed aggregate. `superset/charts/client_processing.py`'s `pivot_table_v2()` post-processor — the code path used for scheduled reports, alerts, and CSV/Excel exports of pivot table charts (`apply_client_processing`, dispatched from `superset/charts/data/api.py` when `result_type == ChartDataResultType.POST_PROCESSED`) — appears to still have the pre-SIP-216 bug class in the default "Actual Values" display mode: - It reads the chart's (largely orphaned, post-SIP-216) `aggregateFunction` form-data field and maps it via `pivot_v2_aggfunc_map` (a dict of pandas reducers, e.g. `pd.Series.median`). - Subtotals are computed at `pivot_table_v2()` lines ~661 and ~699 via `pivot_v2_aggfunc_map[aggfunc](block, axis=1)` / `(..., axis=0)` — i.e. re-aggregating a block of already-per-cell-aggregated values with a second, independent aggregation step. - The correct, database-computed rollups (`rollup_levels`) are only consulted when `percent_mode` is active (`if percent_mode and rollup_levels:` around line 710). In the default, non-percent "Actual Values" mode, subtotals fall through to the pandas re-aggregation path above. This is the same bug class SIP-216 fixed for the live browser view, just not fixed in this separate report/export rendering path (a genuinely separate code path — reports/alerts/exports don't go through the browser's `transformProps.ts`). **I have not yet confirmed this produces incorrect output** — I traced the code but didn't reproduce it live. It needs verification before concluding it's an active correctness bug vs. a harmless vestige (e.g. if `df` at that point always has exactly one row per leaf cell, the re-aggregation may be a no-op for many cases). ### Steps to verify 1. Build a pivot table chart with a non-additive metric (e.g. `AVG(x)` or `COUNT_DISTINCT(x)`), row and/or column subtotals enabled, "Show values as: Actual Values" (not a percent mode). 2. Compare the on-screen (live view) subtotal for a given row/column group against the same subtotal in a scheduled report / CSV or Excel export of the same chart. 3. If they differ, the report/export path is producing incorrect subtotals for non-additive metrics — the same class of bug #41184 fixed for the live view. ### Additional context Related: #41184 (SIP-216, the original fix for the live view), #42761 (follow-up migration for orphaned `aggregateFunction` display values), #42895 (restored MEDIAN/STDDEV_SAMP/VAR_SAMP as metric aggregates, whose own design doc discusses this same totals-computation correctness concern). ### Superset version master / latest-dev ### Checklist - [x] I have searched Superset docs and Slack and didn't find a solution to my problem. - [x] I have searched the GitHub issue tracker and didn't find a similar bug report. - [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
