pipina commented on issue #36682:
URL: https://github.com/apache/superset/issues/36682#issuecomment-3664962179
✅ **Resolved / workaround confirmed**
We were able to resolve the issue.
### Root cause
The email report workflow re-parses CSV data using Pandas with the default
comma delimiter, ignoring the `CSV_EXPORT` configuration.
When a non-default delimiter (e.g. `;`) is used, this causes a Pandas
parsing error and results in HTTP 500 errors for CSV email reports, while GUI
CSV exports continue to work.
### Fix
We patched `superset/charts/client_processing.py` so that CSV parsing during
client-side post processing respects `CSV_EXPORT["sep"]` and
`CSV_EXPORT["decimal"]`.
Specifically:
- import `current_app` from Flask
- read delimiter and decimal from `current_app.config["CSV_EXPORT"]`
- pass them explicitly to `pd.read_csv(...)`
### Bootstrap workaround (Helm / no custom image)
Below is the `bootstrapScript` we used to apply the patch at container
startup without rebuilding the image:
```yaml
bootstrapScript: |
#!/bin/bash
apt-get update && \
apt-get install -y locales && \
apt autoremove -y && \
apt clean
localedef -i sk_SK -f UTF-8 sk_SK.UTF-8 \
rm -rf /var/lib/apt/lists/* && \
uv pip install Pillow \
psycopg2-binary \
cx_Oracle \
sqlalchemy-vertica-python \
pymssql \
redis==4.6.0 \
openpyxl \
playwright \
&& playwright install-deps \
&& PLAYWRIGHT_BROWSERS_PATH=/usr/local/share/playwright-browsers
playwright install chromium
python3 - <<'PY'
from pathlib import Path
import re
path = Path("/app/superset/charts/client_processing.py")
if not path.exists():
raise SystemExit(f"Patch target not found: {path}")
s = path.read_text(encoding="cp1250")
# Ensure current_app import exists
if "from flask import current_app" not in s:
m = re.search(r"^from flask import .*?$", s, flags=re.M)
if m:
insert_at = m.end()
s = s[:insert_at] + "\nfrom flask import current_app" +
s[insert_at:]
else:
s = "from flask import current_app\n" + s
# Replace df = pd.read_csv(StringIO(data)) with config-aware version
pattern = r"^(\s*)df\s*=\s*pd\.read_csv\(\s*StringIO\(data\)\s*\)\s*$"
m = re.search(pattern, s, flags=re.M)
if m and "current_app.config.get(\"CSV_EXPORT\"" not in s:
indent = m.group(1)
replacement = (
indent + "csv_export_config =
current_app.config.get(\"CSV_EXPORT\", dict())\n"
+ indent + "sep = csv_export_config.get(\"sep\", \";\")\n"
+ indent + "decimal = csv_export_config.get(\"decimal\", \",\")\n"
+ indent + "df = pd.read_csv(StringIO(data), sep=sep,
decimal=decimal)"
)
s = re.sub(pattern, replacement, s, flags=re.M)
path.write_text(s, encoding="cp1250")
print("Patched:", path)
PY
if [ ! -f ~/bootstrap ]; then echo "Running Superset with uid {{
.Values.runAsUser }}" > ~/bootstrap; fi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
