L4XB opened a new pull request, #45614:
URL: https://github.com/apache/superset/pull/45614
### SUMMARY
Every CSV export written by `df_to_escaped_csv` doubles each backslash in
the data: chart and SQL Lab CSV downloads, the CSV result format, and CSV
attachments in reports. A Windows path, a regular expression or an escaped JSON
string comes out changed:
```python
df = pd.DataFrame({"path": ["C:\\Users\\ann"]})
df_to_escaped_csv(df, index=False)
# master: 'path\nC:\\\\Users\\\\ann\n' (the file contains C:\\Users\\ann)
# this PR: 'path\nC:\\Users\\ann\n' (the file contains C:\Users\ann)
```
`df_to_escaped_csv` passes `escapechar="\\"` to `DataFrame.to_csv`. #28755
added it as a workaround for pandas-dev/pandas#47871: on Python 3.10, the `csv`
writer raised `need to escape, but no escapechar set` for a value holding a NUL
character. With an escape character set, though, the writer also escapes that
character itself in every field, so each `\` is written as `\\`. The test from
#28755 pins that side effect: the escaped formula `'=cmd\|' /C calc'!A0` is
expected as `'=cmd\\|' /C calc'!A0`.
Superset now requires Python 3.11 (`requires-python = ">=3.11"`), and from
3.11 on the writer writes a NUL without an escape character. I checked this on
3.11, 3.12 and 3.13; the NUL case in `test_df_to_escaped_csv` still passes. The
escape character is therefore set only when the caller asks for
`quoting=csv.QUOTE_NONE`, which has no other way to write a delimiter, quote or
line break. A deployment whose `CSV_EXPORT` sets that keeps the current
behaviour, and `escapechar` passed through `CSV_EXPORT` is respected.
### TESTING INSTRUCTIONS
- `pytest tests/unit_tests/utils/csv_tests.py`: 15 passed.
- New `test_df_to_escaped_csv_keeps_backslashes` checks the default export
(backslashes kept) and `QUOTE_NONE` (escape character still applied). It fails
on `master`.
- `test_df_to_escaped_csv` expects the escaped formula as `'=cmd\|' /C
calc'!A0` again, the way it read before #28755. The formula escaping itself is
unchanged.
- `tests/unit_tests/charts`, `commands/sql_lab`, `common`, `reports`,
`result_set_test.py` and `dataframe_test.py` show no new failures.
`test_df_to_records_preserves_finite_longdouble_and_nulls_nonfinite` and
`test_get_data_json_preserves_browser_numeric_contract` fail the same way on
`master` on this machine (arm64 macOS).
- `ruff check` / `ruff format` (0.9.7) are clean, and `mypy
--check-untyped-defs` / `pylint` report nothing in `superset/utils/csv.py`.
To check by hand, run a SQL Lab query such as `SELECT 'C:\Users\ann' AS
path` (escape the backslash as your database requires) and download the result
as CSV. The file should contain `C:\Users\ann`.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
