david-streamlio opened a new pull request, #71: URL: https://github.com/apache/pulsar-connectors/pull/71
Fixes #70 ### Motivation `DebeziumMysqlSourceTest` and `DebeziumPostgresSourceTest` can hang CI for 45 minutes instead of failing, if their connector delivers fewer CDC records than expected. `AbstractKafkaConnectSource.read()` is a `while (true)` loop whose only `return` yields a record — an empty poll hits `continue`. **It never returns `null`.** Both tests drive it with `while ((record = source.read()) != null)`, escaping only via `break` after 2 records, all inside an Awaitility `untilAsserted` block. If fewer than 2 records arrive, `read()` spins forever, and Awaitility's `atMost(60s)` never fires because `untilAsserted` cannot interrupt a blocked assertion. Neither test declares a `timeOut`, so nothing bounds them. They pass today only because MySQL and Postgres reliably emit their snapshot records. This was observed for real on #58, where the `Tests - Connectors` job ran 45m15s and was **cancelled** — no diagnostics, 45 minutes of runner time gone. ### Modifications For both tests: - Read each record on a single-threaded executor with a per-record deadline (`READ_TIMEOUT_SECONDS = 120`), cancelling the blocked read and failing with a message that names the connector and points at the Debezium logs. - Add `@Test(timeOut = 600_000)` as a backstop. - `shutdownNow()` the executor in cleanup, since a reader may still be blocked in `read()`. - Replace the `while (read() != null)` idiom with an explicit `EXPECTED_RECORDS` loop, so the expectation is stated rather than implied by a `break`. This is the same harness applied in #58. ### Verifying this change Both tests pass locally against real containers: ``` DebeziumMysqlSourceTest > testMysqlCdcEvents PASSED DebeziumPostgresSourceTest > testPostgresCdcEvents PASSED BUILD SUCCESSFUL in 26s ``` The guard itself was verified by forcing under-delivery (expecting 3 records against 2 seeded rows): the fixed harness fails in ~37s with a `TimeoutException`, where the old one hangs indefinitely. ### Related - #58 (MongoDB) carries the same fix - #60 (SQL Server) still has the unbounded pattern and no `timeOut` - #62 (Oracle) and #69 (MariaDB) have a `timeOut` backstop but still read on the test thread Once those land, a follow-up can lift `readOne()` into a shared test base class rather than repeating it per module. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
