GitHub user alberttwong created a discussion: Proposal: Redis as a queryable data source via ADBC and SQLAlchemy
I'd like to propose documenting and integrating Redis as a queryable data source in Superset through the [Redis ADBC driver](https://github.com/alberttwong/redis-adbc-driver), and ask for guidance on the preferred contribution path. ### Use case and architecture The goal is to explore Redis-backed tabular data in SQL Lab and build datasets and charts using the driver's SQL layer. This concerns Redis as an analytics data source, separately from Superset's existing Redis cache and task infrastructure. The connection path is Superset → SQLAlchemy dialect → Python DB-API adapter → native Redis ADBC driver → Redis. Superset would use its existing SQLAlchemy integration; a new ADBC connection subsystem is not required. ### Existing implementation - [Python DB-API adapter](https://github.com/alberttwong/redis-adbc-driver/blob/main/python/adbc_driver_redis/dbapi.py). - [SQLAlchemy dialect](https://github.com/alberttwong/redis-adbc-driver/blob/main/python/adbc_driver_redis/sqlalchemy.py) supporting SQLAlchemy 1.4 and 2.x, PostgreSQL-style SQL compilation, and catalog reflection. - [Redis and Redis TLS engine specs](https://github.com/alberttwong/redis-adbc-driver/blob/main/python/adbc_driver_redis/superset.py), registered through `superset.db_engine_specs` entry points. These inherit PostgreSQL time-grain behavior and map Redis backends to SQLGlot's PostgreSQL parser. - URIs: `redis+adbc://host:6379/0` and `rediss+adbc://host:6379/0` for TLS. - [Setup and limitations](https://github.com/alberttwong/redis-adbc-driver/blob/main/docs/python-integrations.md#sqlalchemy-and-superset). Install the package in every Superset web and SQL Lab worker environment. An [optional Superset 6.1 smoke test](https://github.com/alberttwong/redis-adbc-driver/blob/main/python/tests/test_superset.py) covers registration, schema/table discovery, dataset metadata, grouped SUM queries, daily time-series aggregation, and SQL Lab execution using a temporary metadata database. This describes test coverage, not a fresh compatibility run for this proposal. ### Scope and limitations This is an analytics connector with AUTOCOMMIT only. Rollback cannot undo writes; savepoints and two-phase transactions are unsupported. SQL support comes from the driver's SQL layer. The plugin does not imply complete PostgreSQL compatibility or automatic SQL access to every arbitrary Redis key. Inherited engine-spec capabilities and time grains need review against actual driver capabilities. Before an upstream documentation or engine-spec PR, installation instructions should be verified against package release status, native-library/wheel availability, supported platforms, and the target Superset version. ### Proposed contribution and questions 1. Would maintainers prefer an external `superset.db_engine_specs` plugin with database documentation, or would an in-tree Redis engine spec be appropriate? 2. What compatibility evidence and capability checks should accompany a documentation or engine-spec contribution? 3. Is the PostgreSQL-derived engine spec and parser mapping a suitable starting point, or should specific capabilities be disabled or implemented differently? A focused first contribution could document the external connector and its limitations, with an in-tree engine spec if that is the preferred approach. Implementation tracking: [redis-adbc-driver#273](https://github.com/alberttwong/redis-adbc-driver/issues/273). Upstream submission checklist: [redis-adbc-driver#343](https://github.com/alberttwong/redis-adbc-driver/issues/343). GitHub link: https://github.com/apache/superset/discussions/45536 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
