mistercrunch commented on issue #26646: URL: https://github.com/apache/superset/issues/26646#issuecomment-2730824762
This is an interesting problem, but to do it right, we'd need a deeper refactor to unify the way we interact with analytics databases across SQL Lab and Explore. Right now, SQL Lab has a bunch of custom and database-specific logic for executing blocks of SQL, while Explore doesn’t have this capability at all. Both parts of the app have their own approaches to things like SQL parsing, enforcing data access rules, caching, and handling different database backends, leading to code duplication and divergence. If we’re adding block execution logic to Explore, it seems like the right move would be to centralize this logic instead of re-implementing it there. Ideally, we push most of this down into superset/db_engine_specs/, making it the core layer for SQL parsing and execution. Both SQL Lab and Explore could then interact with it through a shared, higher-level API. I realize this is a bigger effort than the immediate SIP, but it feels like the right path forward to avoid accumulating more tech debt. This might also fit into a broader push to formalize a proper Data Access Layer in Superset—a standalone Python app that manages all this logic cleanly. That’s probably its own SIP, but this work could be a step in that direction. Some pointers to the relevant diverging/duplicated code: - [handling multiple block in sql lab](https://github.com/apache/superset/blob/master/superset/sql_lab.py#L502-L537) - needs refactor towards `db_engine_specs/base.py` - [Database.get_df](https://github.com/apache/superset/blob/master/superset/models/core.py#L677-L692) - this runs pretty deep in the call stack, and also use db_engine_spec down the call stack, but has a lot of Explore-specific logic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
