dosubot[bot] commented on issue #36682:
URL: https://github.com/apache/superset/issues/36682#issuecomment-3665725787
<!-- Answer -->
The reason your CSVs are still UTF-8 is that the post-processing step in
`apply_client_processing` writes CSV data using pandas defaults, ignoring the
encoding from your config. To fix this, you need to patch that function so it
uses the encoding from `CSV_EXPORT` when writing the CSV.
Here’s how to update the relevant part of
`superset/charts/client_processing.py`:
```python
from flask import current_app
# ... inside apply_client_processing, in the CSV writing block:
csv_export_config = current_app.config.get("CSV_EXPORT", {})
sep = csv_export_config.get("sep", ";")
decimal = csv_export_config.get("decimal", ",")
encoding = csv_export_config.get("encoding", "utf-8")
buf = BytesIO()
processed_df.to_csv(buf, index=show_default_index, sep=sep, decimal=decimal,
encoding=encoding)
buf.seek(0)
query["data"] = buf.getvalue().decode(encoding)
```
This ensures the exported CSV uses your specified encoding (cp1250),
separator, and decimal. You can adapt your bootstrap patch to replace the
writing block as well as the reading block.
After patching, test with cp1250-specific characters and verify the encoding
using `file` on Linux and by opening the file in Excel on Windows. If you run
into pandas limitations with cp1250, try with a small sample first.
This change will make both GUI downloads and email attachments respect your
encoding config, matching the rest of Superset’s export logic
[reference](https://github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/charts/client_processing.py#L311-L388).
<!-- Dosu Comment Footer -->
*To reply, just mention [@dosu](https://go.dosu.dev/dosubot).*
---
<sup>How did I do?
[Good](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=great_response)
|
[Irrelevant](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=irrelevant_answer)
|
[Incorrect](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=incorrect_sources)
|
[Verbose](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=too_verbose)
|
[Hallucination](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=hallucination)
| [Report
🐛](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=bug_report)
|
[Other](https://app.dosu.dev/response-feedback/155e92d9-b722-40e2-a59a-bb0b03f7c809?feedback_type=other)</sup>
[](https://app.dosu.dev/a28d3c7e-a9d3-459e-9fb6-3a6f9ff4f357/ask?utm_source=github) [](https://cursor.com/link/prompt?text=The%20reason%20your%20CSVs%20are%20still%20UTF-8%20is%20that%20the%20post-processing%20step%20in%20%60apply_client_processing%60%20writes%20CSV%20data%20using%20pandas%20defaults%2C%20ignoring%20the%20encoding%20from%20your%20config.%20To%20fix%20this%2C%20you%20need%20to%20patch%20that%20function%20so%20it%20uses%20the%20encoding%20from%20%60CSV_EXPORT%60%20when%20writing%20the%20CSV.%0A%0AHere%E2%80%99s%20how%20to%20update%20the%20relevant%20part%20of%20%60superset/charts/client_processing.py%60%3A%0A%0A%60%60%60python%0Afrom%20flask%20import%20current_app%0A%0A%23%20...%20inside%20apply_client_processing%2C%20in%20the%20CSV%20writing%20block%3A%0Acsv_export_config%20%3D%20current_app.config.get%28%22CSV_EXPORT%22%2C%20%7B%7D%29%0Asep%20%3D%20csv
_export_config.get%28%22sep%22%2C%20%22%3B%22%29%0Adecimal%20%3D%20csv_export_config.get%28%22decimal%22%2C%20%22%2C%22%29%0Aencoding%20%3D%20csv_export_config.get%28%22encoding%22%2C%20%22utf-8%22%29%0A%0Abuf%20%3D%20BytesIO%28%29%0Aprocessed_df.to_csv%28buf%2C%20index%3Dshow_default_index%2C%20sep%3Dsep%2C%20decimal%3Ddecimal%2C%20encoding%3Dencoding%29%0Abuf.seek%280%29%0Aquery%5B%22data%22%5D%20%3D%20buf.getvalue%28%29.decode%28encoding%29%0A%60%60%60%0A%0AThis%20ensures%20the%20exported%20CSV%20uses%20your%20specified%20encoding%20%28cp1250%29%2C%20separator%2C%20and%20decimal.%20You%20can%20adapt%20your%20bootstrap%20patch%20to%20replace%20the%20writing%20block%20as%20well%20as%20the%20reading%20block.%0A%0AAfter%20patching%2C%20test%20with%20cp1250-specific%20characters%20and%20verify%20the%20encoding%20using%20%60file%60%20on%20Linux%20and%20by%20opening%20the%20file%20in%20Excel%20on%20Windows.%20If%20you%20run%20into%20pandas%20limitations%20with%20cp1250%2C%20try%20with%2
0a%20small%20sample%20first.%0A%0AThis%20change%20will%20make%20both%20GUI%20downloads%20and%20email%20attachments%20respect%20your%20encoding%20config%2C%20matching%20the%20rest%20of%20Superset%E2%80%99s%20export%20logic%C2%A0%5Breference%5D%28https%3A//github.com/apache/superset/blob/829e4d92d91ceae4b43b1ed3b063ffe45377799c/superset/charts/client_processing.py%23L311-L388%29.) [](https://go.dosu.dev/discord-bot) [](https://twitter.com/intent/tweet?text=%40dosu_ai%20helped%20me%20solve%20this%20issue!&url=https%3A//github.com/apache/superset/issues/36682)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
