david-streamlio opened a new pull request, #69: URL: https://github.com/apache/pulsar-connectors/pull/69
Fixes #65 ### Motivation Debezium ships a MariaDB source connector (`io.debezium:debezium-connector-mariadb`, present in the `debezium-bom` at our pinned version), but this repository did not wrap it. We wrapped 5 of Debezium's 12 source connectors; MariaDB was the most valuable gap: - Debezium split the MariaDB connector out of the MySQL connector, and both now build on the shared `debezium-connector-binlog` base — so the existing `debezium/mysql` module is a near drop-in template. - We already ship a MariaDB **JDBC sink** (integration test added in #59), so this completes the story for that database. ### Modifications New `debezium/mariadb` module (`pulsar-io-debezium-mariadb`), mirroring `debezium/mysql`: - `DebeziumMariaDbSource` extending `DebeziumSource`, pinning Debezium's `MariaDbConnector` / `MariaDbConnectorTask` - Service descriptor (`META-INF/services/pulsar-io.yaml`) and sample source config - Version catalog entry, `settings.gradle.kts` include, and wiring into `distribution/io`, `docker/pulsar-all`, the assembly XML, and `docs` ### Test suite **Unit tests** (`DebeziumMariaDbSourceConfigTest`, 7 cases, no database required) cover the connector/task class contract, which is otherwise a silent misconfiguration that only surfaces at runtime: - both keys are defaulted when absent, and accepted when explicitly set to the matching value - a mismatched connector or task class is rejected — specifically guarding against pointing the MariaDB source at the **MySQL** connector, an easy mistake now that the two share a binlog base - both class names resolve via `Class.forName`, proving the string literals name real classes on the runtime classpath **Integration test** (`DebeziumMariaDbSourceTest`) runs the source against a real `mariadb:11.4` server plus a `PulsarContainer` for Debezium's schema history, seeds two rows, and asserts the initial snapshot emits CDC records. ### Verifying this change ``` DebeziumMariaDbSourceConfigTest > 7 tests PASSED DebeziumMariaDbSourceTest > testMariaDbCdcEvents PASSED BUILD SUCCESSFUL in 24s ``` NAR packaging verified: `pulsar-io-debezium-mariadb-4.3.0-SNAPSHOT.nar` builds. ### Notes - Uses the `GenericContainer` pattern rather than Testcontainers' `MariaDBContainer`: `debezium-bom` pins Testcontainers core to 2.x, while the specialized modules still target 1.21.x and reference shaded classes removed in 2.x (see #62). - The integration test carries a `timeOut` because `source.read()` blocks — without it, a connector that never emits would hang until the CI job's own timeout rather than failing. - The MariaDB 3.x JDBC driver (which arrives transitively with the connector, so the NAR has it) only registers the `jdbc:mariadb:` scheme and no longer handles `jdbc:mysql:` URLs. - MariaDB is single-partition, so it does not hit the `task.id` gap described in #61. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
