sha174n commented on PR #42412: URL: https://github.com/apache/superset/pull/42412#issuecomment-5092832638
Thanks for catching this. Routing through `get_raw_connection()` to restore impersonation / SSH tunnel / OAuth2 parity with the non-streaming paths is the right call, and adding `mutate_sql_based_on_config()` is clearly needed. One thing to confirm before this lands: the previous path set `execution_options(stream_results=True)` (a SQLAlchemy server-side cursor), and the new raw-DBAPI `cursor.execute()` + `fetchmany()` drops it. For drivers that buffer the full result set client-side on `execute()`, that would defeat the streaming this path exists for and risk high memory use on the large (100k+ row) exports it targets. Could you confirm the raw cursor still streams server-side for the target engines (Trino/Presto), or add the server-side cursor / `arraysize` config to preserve it? Separately, CI is currently red (`unit-tests`, `test-sqlite`, `docker-build`), which looks like the tests/build need updating for the raw-connection change. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
