AryaKetanShCt opened a new issue, #42926: URL: https://github.com/apache/superset/issues/42926
### Bug description A chart is persisted twice: `slices.params` holds the chart's settings, and `slices.query_context` holds a query built from those settings. Explore rebuilds the query from `params` at every render and never reads `query_context`. `query_context` is written only when a user saves the chart, and nothing rewrites it on upgrade. Every consumer that is **not** the browser replays that stored query verbatim: - `GET /api/v1/chart/<id>/data/` — `superset/charts/data/api.py` does `json.loads(chart.query_context)` - alerts and reports, thumbnails, cache warm-up, CSV/Excel export, which go through the same endpoint - the MCP server's chart-data tools So the stored query ages while the query engine moves on, and a chart last saved years ago hands an old query to a current engine. The result is a 500 on all of those paths, while the very same chart renders perfectly in Explore. This makes it look like a data or permission problem rather than a schema-drift problem. This is the same family as #33152 (*"Viz migrations only migrate the form_data and not the query object. This is problematic as backend processes such as cache warmup or alerts & reports use the saved query object"*) and #31872, but it needs no import and no viz migration to trigger — only time. #### Symptom 1 — a removed post-processing option `QueryObject.exec_post_processing` passes the stored options straight through: ```python df = getattr(pandas_postprocessing, operation)(df, **options) ``` `pivot` used to take `flatten_columns` and `reset_index`. Flattening later became its own `flatten` operation and the parameters were removed, but charts saved before that still carry them in their stored `query_context`. Replaying one gives: ``` TypeError: pivot() got an unexpected keyword argument 'flatten_columns' ``` There is no migration for the stored query, so this is permanent for those charts until somebody opens each one in Explore and re-saves it. There is also a second defect that hides this from any introspection-based fix. `validate_column_args` in `superset/utils/pandas_postprocessing/utils.py` returns `def wrapped(df, **options)` and does not use `functools.wraps`, so: ``` >>> inspect.signature(pivot) (df: object, **options: object) -> object ``` Ten operations use that decorator (`aggregate`, `compare`, `contribution`, `cum`, `diff`, `pivot`, `rename`, `rolling`, `select`, `sort`). All of them report a `**kwargs` signature, lose their `__name__` and `__doc__`, and cannot be introspected. `inspect.unwrap` does not help, because without `wraps` there is no `__wrapped__`. #### Symptom 2 — a time series with no time column Charts saved by an older frontend store `is_timeseries: true` in the query while the temporal column survives only in `params` as `granularity_sqla`. `superset/models/helpers.py` then raises: ``` Datetime column not provided as part table configuration and is required by this type of chart ``` Again, correct in Explore, 500 everywhere else. ### How to reproduce 1. Take a chart saved by an older Superset (in our instance: an `echarts_timeseries_bar` saved 2023-09 and a `big_number` saved 2023-02). No import, no migration, no edit is needed. 2. Open it in Explore — it renders correctly. 3. Call `GET /api/v1/chart/<id>/data/` for the same chart, or attach an alert to it. 4. Symptom 1 or symptom 2 occurs, depending on what the stored query contains. To confirm the second defect on any version: ```python import inspect from superset.utils.pandas_postprocessing import pivot print(inspect.signature(pivot)) # (df, **options) — the wrapper, not pivot print(pivot.__name__) # 'wrapped' ``` ### Expected results A stored `query_context` written by an older version stays runnable, or fails with a message that names the cause. A chart that renders in Explore should not 500 on the chart-data endpoint. The post-processing operations should report their own signature and name. ### Actual results `TypeError: pivot() got an unexpected keyword argument 'flatten_columns'`, or `Datetime column not provided as part table configuration and is required by this type of chart`, on every non-Explore path, permanently, for charts that display correctly in the browser. ### Screenshots/recordings _No response_ ### Superset version master / latest-dev ### Python version 3.11 ### Node version I don't know ### Browser Not applicable (server side) ### Additional context Confirmed on current master (`3b164e4`): `charts/data/api.py` still reads the stored query, `pivot()` still has no `flatten_columns`, the raise in `models/helpers.py` is unchanged, and `validate_column_args` still has no `functools.wraps`. Superset 5.0.0 also has no `flatten_columns`, so symptom 1 is not new. A forced refresh does not help. `force` only bypasses the result cache; the stored query is the input that builds the SQL. Measured on a 6.1.0 instance with ~2000 `echarts_timeseries_bar` and ~1300 `big_number` charts: charts saved in 2023 fail, charts saved in 2026 succeed, which matches the age of the stored query rather than the visualisation type. I have a fix for symptom 1 and for the decorator, and will open a PR that links this issue. Symptom 2 is left out of that PR on purpose: `_apply_granularity` on master has gained its own inference path, so the intended behaviour there is worth a maintainer's opinion before I send code. ### Checklist - [x] I have searched Superset docs and Slack and didn't find a solution to my problem. - [x] I have searched the GitHub issue tracker and didn't find a similar bug report. - [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
