rusackas opened a new pull request, #43668:
URL: https://github.com/apache/superset/pull/43668
### SUMMARY
Stacks on #43662. Adds 6 more dialects to the testcontainers-based
`db_engine_specs` suite, selected from a research pass over ~11 candidates
(Solr, IoTDB, TDengine, Parseable, and Dremio were dropped -- each needs a
schema-bypass test design or deeper investigation, not a quick win). All six
extras already existed in `pyproject.toml`, so this is purely test wiring.
`oceanbase` and `vertica` run `nightly_only: true` (the gating mechanism
added in #43662): OceanBase bootstraps a distributed-style cluster even in
single-node mode, and Vertica Community Edition has a well-documented ~12GB RAM
floor to even start. `databend`/`risingwave`/`firebird`/`ydb` run on every PR
like the rest of this suite.
Real quirks found along the way:
- `oceanbase_py` pins `sqlalchemy-utils<0.39`, which conflicts outright with
Superset's own `sqlalchemy-utils==0.42.1` pin. Kept out of the baseline dev
install (same treatment as `db2`'s `ibm-db-sa`) and installed on demand with
`--no-deps` for just its own CI leg -- it never actually imports
`sqlalchemy_utils` itself, so the version mismatch is inert at runtime.
- RisingWave's storage engine checkpoints asynchronously: a `SELECT`
immediately after `INSERT` can see zero rows without an explicit `FLUSH`
(confirmed against a real instance). Uses the shared `_pagination.py` helper's
`after_insert` hook, originally added for CrateDB, to handle this.
- `sqlalchemy-firebird`'s driver is a pure-Python ctypes wrapper (its wheel
is `py3-none-any`, confirmed by downloading it directly) that dynamically loads
the native `libfbclient` from the host rather than bundling it -- CI installs
that system package on demand, only for the firebird leg.
- YDB needed three real fixes to make a generic `DockerContainer` usable at
all: (1) its gRPC client does endpoint discovery and reconnects to whatever the
server reports, which by default is the container's own internal Docker
hostname -- fixed by binding the same port on host and container and
advertising `localhost` as the container's own hostname; (2) the gRPC port
opens before storage pools finish initializing, so the fixture retries a real
`metadata.create_all()` probe rather than trusting the open port; (3) YDB
rejects DDL inside an explicit transaction, which only affects a raw
`text("CREATE TABLE...")`, not `metadata.create_all()`'s own DDL execution path.
- Small side finding, not acted on here: `FirebirdEngineSpec` sets
`limit_method = LimitMethod.FETCH_MANY` with a comment claiming Firebird "uses
FIRST to limit" -- stale relative to the modern driver, which compiles real
`ROWS`-based pagination correctly. Noted in the test docstring as a possible
follow-up.
### TESTING INSTRUCTIONS
CI: `.github/workflows/testcontainers.yml` runs each per-PR dialect in its
own matrix job; `oceanbase`/`vertica` only run on the nightly cron or a manual
`workflow_dispatch`.
Locally verified (databend, risingwave, ydb -- each iterated against a real
container until green): `pytest -m testcontainers
tests/testcontainers/db_engine_specs/test_databend.py
tests/testcontainers/db_engine_specs/test_risingwave.py
tests/testcontainers/db_engine_specs/test_ydb.py`. Firebird's DDL/pagination
compilation was verified offline (no live connection possible on this machine
-- `firebird-driver` needs a native client library with no macOS install path
here); CI's `libfbclient2` system package install gives it a real client
library to actually connect. OceanBase and Vertica weren't pulled locally at
all given their heavier resource footprint -- both rely on a manual
`workflow_dispatch` run for first real verification, matching the
`nightly_only` gating.
### ADDITIONAL INFORMATION
- [ ] Has associated issue:
- [ ] Required feature flags:
- [ ] Changes UI
- [ ] Includes DB Migration (follow approval process in
[SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [ ] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
