danlewis85 opened a new issue, #32862: URL: https://github.com/apache/superset/issues/32862
### Bug description This bug describes the cause for a bug previously described in #22467 and later closed without explicit resolution. It offers a possible patch, but not a full fix. This is the issue I encountered in superset v. 4.1.1 using Duckdb. In the superset database connection (edit database --> Advanced --> Other additional settings), I wanted to pass some engine parameters, however this was failing. What I found is that the parameters weren't being decoded correctly as part of the `get_extra_params()` method of the [base database engine class](https://github.com/apache/superset/blob/6f69c84d10c3bf9f4bfa20ae876d1dfb7cc695e3/superset/db_engine_specs/base.py#L2047). This meant that when [duckdb.py calls that method](https://github.com/apache/superset/blob/45ea11c1b65887755f30e5945ea280abc0847929/superset/db_engine_specs/duckdb.py#L240) to get the extra parameters, you don't get a properly formed nested dictionary, instead you get a dictionary with the first key and then all other info as a string for the value. So, if my input was: `{"engine_params":{'connect_args': {'read_only': True}}}` superset is passing back: `{"engine_params":"{'connect_args': {'read_only': False}}"}` (Note the extra quotes around the value) It should be passing: `{"engine_params":{'connect_args': {'read_only': True}}}` This means that duckdb.py is failing on its `get_extra_params()` method because it's trying to operate on a string instead of a dictionary. One solution that worked for me was to fix the immediate problem - this worked for my use case: ``` # /superset/db_engine_specs/duckdb.py from ast import literal_eval @staticmethod def get_extra_params( database: Database, source: QuerySource | None = None ) -> dict[str, Any]: """ Add a user agent to be used in the requests. """ extra: dict[str, Any] = BaseEngineSpec.get_extra_params(database) # >>> Start new code for key in extra.key(): try: extra[key] = literal_eval(extra[key]) except ValueError: pass # >>> End new code engine_params: dict[str, Any] = extra.setdefault("engine_params", {}) connect_args: dict[str, Any] = engine_params.setdefault("connect_args", {}) config: dict[str, Any] = connect_args.setdefault("config", {}) custom_user_agent = config.pop("custom_user_agent", "") delim = " " if custom_user_agent else "" user_agent = get_user_agent(database, source) user_agent = user_agent.replace(" ", "-").lower() user_agent = f"{user_agent}/{VERSION_STRING}{delim}{custom_user_agent}" config.setdefault("custom_user_agent", user_agent) return extra ``` However, as `literal_eval` is just turning a presumed string into its representation (or passing if not a string) the best solution is probably to fix the root cause problem of the malformed json being passed in the first place. Unfortunately, I don't know the code base well enough to know where to make that change. Also, I assume that this could be a problem for databases other than duckdb. Given that it appears to be a problem from the Database object and base database engine class. ### Screenshots/recordings _No response_ ### Superset version master / latest-dev ### Python version 3.9 ### Node version 16 ### Browser Chrome ### Additional context _No response_ ### Checklist - [x] I have searched Superset docs and Slack and didn't find a solution to my problem. - [ ] I have searched the GitHub issue tracker and didn't find a similar bug report. - [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
