david-streamlio opened a new pull request, #62:
URL: https://github.com/apache/pulsar-connectors/pull/62
Fixes #44
### Motivation
The `debezium/oracle` module (`DebeziumOracleSource`) had no tests. This PR
adds a Testcontainers integration test that exercises the connector end-to-end
against a real Oracle database, mirroring the shape of the existing
`DebeziumMysqlSourceTest`.
Oracle is the heaviest Debezium variant to test: LogMiner-based CDC requires
archive log mode, supplemental logging, a common (CDB-wide) user with an
extensive grant set, and PDB-aware connector configuration. All of that is
handled inside the test via a container init script.
### Modifications
- New `DebeziumOracleSourceTest` in `debezium/oracle`:
- Starts `gvenzl/oracle-free:23-slim-faststart` (the image family
Debezium's own test infrastructure uses) plus a `PulsarContainer` for the
schema-history topic.
- A SYSDBA init script (`src/test/resources/debezium-oracle-setup.sql`,
copied to `/container-entrypoint-initdb.d/`) enables archive log mode
(`SHUTDOWN IMMEDIATE` / `STARTUP MOUNT` / `ALTER DATABASE ARCHIVELOG`), enables
minimal supplemental logging, creates `LOGMINER_TBS` tablespaces in CDB and
PDB, creates `c##dbzuser` with the LogMiner grant set from the Debezium 3.x
documentation, and seeds a `DEBEZIUM.PRODUCTS` table (2 rows) in `FREEPDB1`
with `SUPPLEMENTAL LOG DATA (ALL) COLUMNS`. The gvenzl entrypoint runs init
scripts before printing `DATABASE IS READY TO USE!`, so the container wait
strategy also covers the DB restart.
- Opens `DebeziumOracleSource` with `database.dbname=FREE`,
`database.pdb.name=FREEPDB1`, `table.include.list=DEBEZIUM.PRODUCTS`,
`log.mining.strategy=online_catalog`, and a Pulsar-backed schema history, then
asserts the initial snapshot emits at least 2 records.
- `debezium/oracle/build.gradle.kts`: adds `testcontainers`,
`testcontainers-pulsar`, and `pulsar-client` test dependencies.
Notes on two dependency findings:
- **Oracle JDBC driver**: no extra test-scoped driver is needed.
`io.debezium:debezium-connector-oracle:3.4.2.Final` pulls
`com.oracle.database.jdbc:ojdbc11:21.15.0.0` (plus
`orai18n`/`xdb`/`xmlparserv2`) transitively onto the runtime classpath, so the
driver also ends up in the NAR. No runtime-driver gap in this module as built
today.
- **Why `GenericContainer` instead of
`org.testcontainers.oracle.OracleContainer`**: `debezium-bom:3.4.2.Final`
constrains Testcontainers core to 2.0.2, while the
`org.testcontainers:oracle-free` module only exists up to 1.21.4 and references
shaded classes removed in core 2.x (`NoClassDefFoundError:
org.testcontainers.shaded.org.apache.commons.lang3.StringUtils` at
`OracleContainer.withPassword`). Using `GenericContainer` against core 2.0.2
avoids the mixed-version breakage and matches the approach already used by
`DebeziumMysqlSourceTest`.
### Verifying this change
Run locally (Docker required):
```
./gradlew :debezium:pulsar-io-debezium-oracle:test
```
Local result (macOS/arm64, Docker Desktop): **PASSED** in ~37s once the
image was pulled; the test received snapshot records with keys `{"ID":1}` and
`{"ID":2}`. The init script was additionally verified standalone:
`V$DATABASE.LOG_MODE=ARCHIVELOG`, `SUPPLEMENTAL_LOG_DATA_MIN=YES`, `c##dbzuser`
can connect to `FREEPDB1` and sees the 2 seeded rows, and the container logs
contain no `ORA-`/`SP2-` errors. `:debezium:pulsar-io-debezium-oracle:check`
(spotless/checkstyle) passes.
CI-runner risk worth calling out explicitly: the
`gvenzl/oracle-free:23-slim-faststart` image is large (~1.6 GB) and Oracle Free
wants ~2 GB of memory; the init script also restarts the database once to
enable archive log mode. Startup on a standard GitHub-hosted runner will be
minutes, not seconds — the test uses a 6-minute container startup timeout, a
5-minute Awaitility window for the snapshot, and a 10-minute overall test
timeout. If the runner is memory-constrained alongside other container tests,
this test will be the first to feel it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
