sadpandajoe opened a new pull request, #45657: URL: https://github.com/apache/superset/pull/45657
### SUMMARY The nightly Python-next (3.13) canary fails on `tests/unit_tests/sql/execution/test_async_results.py::test_deep_nested_sidecar_falls_back_to_arrow[True]` and `[False]` (2 failed, 30501 passed). The main Python-Unit job only runs 3.11, so this does not show up on PR/push CI. Root cause: the test was added in #45348 (f0cfaff788). It builds a 550-level nested list and expects `_encode_nested_value` (`superset/sql/execution/serialization.py`) to raise `RecursionError`, which makes the async writer drop the nested sidecar and fall back to the Arrow string column. That holds on Python 3.11 only. There, every list comprehension adds a stack frame, so the encoder hits the default limit of 1000 at about 497 levels. Python 3.12+ inlines comprehensions (PEP 709), so the encoder handles about 996 levels. On 3.13 the sidecar is written and decoded, and the async result holds the real nested list. That matches the synchronous `to_pandas_df()` result, so the product behaves correctly. The test's expected Arrow string differs. Raising the depth does not fix this. On 3.11, building the `SupersetResultSet` itself raises `RecursionError` at about 960+ levels, while 3.12+ needs more than 996 to reach the fallback, so no single depth works on every supported interpreter. This change patches `_encode_nested_value` to raise `RecursionError` and asserts it was called. The test therefore exercises the fallback deterministically on every interpreter, and the assertion that the async result equals the Arrow fallback frame is unchanged. No product code changes. ### TESTING INSTRUCTIONS 1. Create Python 3.13 and 3.11 environments and install `requirements/development.txt`, the same way the nightly canary does for 3.13. 2. On master, run `pytest tests/unit_tests/sql/execution/test_async_results.py -k test_deep_nested_sidecar_falls_back_to_arrow` under 3.13: both parametrizations fail with `DataFrame.iloc[:, 0] (column name="value") values are different (100.0 %)`. Under 3.11 they pass. 3. On this branch, run the same command under both interpreters: both parametrizations pass. 4. Optional: remove `RecursionError` from the `except` tuple in `serialize_nested_columns` (`superset/sql/execution/serialization.py`). The test now fails on both interpreters, which shows it still guards the fallback. ### ADDITIONAL INFORMATION - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [ ] Introduces new feature or API - [ ] Removes existing feature or API Notes: - Related canary tracking issue: #45429. - Because the encoder error is now forced by the patch, the 550-deep value no longer drives the behavior; any nested value would take the same path. It is kept to leave the test inputs unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
