nytai opened a new pull request, #45254: URL: https://github.com/apache/superset/pull/45254
### SUMMARY Druid serializes NaN doubles as the JSON string `"NaN"`. A metric column that mixes floats with `"NaN"` can't be converted to an Arrow array, so `SupersetResultSet` stringifies every value in it (`0.034218042` -> `'0.034218042'`). `df_columns_to_num` is meant to recover string-encoded numerics (added for ClickHouse), but it rejects these columns: `pd.to_numeric(errors="coerce")` turns `"NaN"` into null, and the "no non-null value was lost" check treats that as a non-numeric value. The column stays as strings, and pandas post-processing (pivot, rolling, ...) fails with: ``` Could not convert string '0.034218042' to numeric Could not convert string 'NaN' to numeric ``` This change treats `"NaN"` strings as null in that check, so these columns are converted. Columns with other non-numeric values are still left unchanged. Legacy `viz.py` charts weren't affected because they coerce metrics unconditionally, so this surfaces when Druid charts are migrated to the ECharts / chart-data API path. ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A ### TESTING INSTRUCTIONS - `pytest tests/unit_tests/common/test_dataframe_utils.py` - Against Druid: build a chart with a metric that yields NaN for some groups (e.g. `SUM(a) / SUM(b)` where `SUM(b)` is 0) plus a post-processing step; it errors before the change and renders after. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
