This is an automated email from the ASF dual-hosted git repository. maximebeauchemin pushed a commit to branch refactor_json in repository https://gitbox.apache.org/repos/asf/superset.git
commit 28118e48e056ddb7c3198db257768eff5a7c3afb Author: Maxime Beauchemin <[email protected]> AuthorDate: Mon Apr 29 09:21:43 2024 -0700 fix: use pessimistic json encoder in SQL Lab An issue reported in https://github.com/apache/superset/issues/28001 shows SQLLab failing hard when a database driver returns a type that can't be decoded for serialization. Switching to this pessimistic helper function for json serialization will prevent a hard failure, but will obfuscate the data for the specific cells where things can't be serialized. Not being in a situation where I can reproduce the issue, I'm operating with the stacktrace, meaning I can't really handle the specific cell, but can prevent SQL Lab from failing hard on a single cell issue. Few related notes: - maybe there's something around the driver being in utf8 mode of some kind. In theory we could say that driver / superset should agree on utf8 for things to work - driver should probably giving us a `bytes` type from a `VARBINARY` and from the stacktrace it appears to be something else, presumably `str`, though that's unclear - could be good to refactor out everything json-related out of superset/utils/core.py and into a new superset/utils/json.py, and make sure we always use thesame utility functions throughout the codebase as opposed to raw `import json` across the codebase. From my understanding there's some magic around simplejson injecting itself everywhere that happens too. --- superset/sqllab/api.py | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/superset/sqllab/api.py b/superset/sqllab/api.py index 24c04dfe5e..cbd96f9562 100644 --- a/superset/sqllab/api.py +++ b/superset/sqllab/api.py @@ -340,13 +340,15 @@ class SqlLabRestApi(BaseSupersetApi): key = params.get("key") rows = params.get("rows") result = SqlExecutionResultsCommand(key=key, rows=rows).run() + + payload = json.dumps( + result, + default=utils.pessimistic_json_iso_dttm_ser, + ignore_nan=True, + ) # return the result without special encoding return json_success( - json.dumps( - result, - default=utils.json_iso_dttm_ser, - ignore_nan=True, - ), + payload, 200, )
