rusackas commented on issue #42840:
URL: https://github.com/apache/superset/issues/42840#issuecomment-5246461449
Dug into this. It's not actually a Superset bug — Superset's own dict/JSON
stringification (`stringify_values` in `result_set.py`) already handles values
like this correctly, verified against the existing test suite. The value is
already `None` by the time it reaches Superset, because `pyathena` converts it
internally inside `cursor.fetchall()`.
Traced it to `pyathena/converter.py`'s `_to_map()`. It has a safety check
meant to skip genuinely complex/nested map values, but it's too broad: it bails
to `None` if the map's *value* contains any of `()[]`, even when those are just
plain characters inside a string rather than real Athena map/array syntax.
Confirmed directly:
```python
_to_map('{key={"key2":[string]}}') # -> None
_to_map('{key=hello world}') # -> {'key': 'hello world'} (control,
works)
```
Workaround in the meantime (same one that worked for #37363): `CAST(payload
AS JSON)` in the query, which sidesteps pyathena's native-format parser
entirely.
Going to see if there's a reasonable upstream fix and open a PR against
pyathena directly, since that's where this actually needs to be fixed. Will
link it here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
