rusackas opened a new pull request, #43640:
URL: https://github.com/apache/superset/pull/43640
### SUMMARY
Stacked on #43502 (the 7-dialect testcontainers pilot) — adds 5 more
dialects from the docs-supported list: `mariadb`, `timescaledb`, `yugabytedb`,
`monetdb`, `mongodb`. All 5 verified locally against real containers except
`mariadb` (mysqlclient has a pre-existing, unrelated native-library issue on
this dev machine's Homebrew setup — same accepted CI-only-verification pattern
already used for crate/mssql/db2 in the base branch).
### Wire-compatible dialects (near-zero new container code)
`mariadb`, `timescaledb`, and `yugabytedb` are all wire-compatible with an
existing base dialect, so they reuse testcontainers'
`MySqlContainer`/`PostgresContainer` classes pointed at a different image
rather than needing new container-class wiring — `mariadb:11`,
`timescale/timescaledb:latest-pg16`, and `yugabytedb/yugabyte:latest`
respectively.
`yugabytedb` needed one adjustment though: `PostgresContainer`'s built-in
readiness check execs `psql`, which the yugabyte image doesn't ship (only its
own `ysqlsh`). Used a generic `DockerContainer` instead, started via `yugabyted
start --background=false` and waiting on yugabyted's own final startup log line
("Data placement constraint successfully verified").
### monetdb (no native testcontainers module)
Uses a generic `DockerContainer` with the documented `MDB_*` environment
variables and a log-based wait strategy. Publishes an amd64-only image —
confirmed it actually runs under Rosetta/QEMU emulation on Apple Silicon
(unlike CrateDB, which fails outright on its x86-64-v3 CPU instruction-set
requirement even under emulation).
### mongodb (two real bugs found writing this one)
MongoDB is schemaless, and Superset talks to it via `pymongosql`, a
SQL-to-MongoDB translation layer requiring a `?mode=superset` query param.
Documents get inserted via the native pymongo driver rather than SQL INSERT,
matching how Superset actually encounters MongoDB in practice and avoiding any
assumption about pymongosql's own INSERT/DDL support.
Two bugs surfaced empirically while wiring this up:
`MongoDbContainer.get_connection_url()` has no database path segment or query
string at all, so naively appending `"&mode=superset"` glued directly onto the
port number instead of starting a query string — fixed by building the full URL
manually. And the root user `MongoDbContainer` creates lives in the `admin`
database, so connecting with a different default database in the URL path fails
authentication outright unless `authSource=admin` is specified explicitly.
Confirmed `pymongosql` actually supports `OFFSET` (it maps to MongoDB's
native `skip`), unlike Elasticsearch's SQL layer in the base PR, which has no
`OFFSET` support at all — so this one keeps the full `LIMIT`/`OFFSET`
pagination test rather than the `LIMIT`-only version.
### New extras added
`monetdb` (`sqlalchemy-monetdb`, `pymonetdb`) and `mongodb` (`pymongosql`)
are genuinely new pyproject.toml extras. `mariadb`, `timescaledb`, and
`yugabytedb` are added as thin aliases (`apache-superset[mysql]` /
`apache-superset[postgres]`) since they need no new packages —
`mysqlclient`/`psycopg2-binary` are already pulled in via the existing
`mysql`/`postgres` extras.
### TESTING INSTRUCTIONS
```
pip install -r requirements/development.txt
pytest -m testcontainers
tests/testcontainers/db_engine_specs/test_monetdb.py
tests/testcontainers/db_engine_specs/test_mongodb.py
tests/testcontainers/db_engine_specs/test_timescaledb.py
tests/testcontainers/db_engine_specs/test_yugabytedb.py -v
```
8 passed locally. `mariadb` needs CI (Linux) to verify due to the local
mysqlclient issue noted above.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API (5 more testcontainers dialects)
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
