wombatu-kun opened a new pull request, #20312: URL: https://github.com/apache/hudi/pull/20312
### Describe the issue this Pull Request addresses Closes #20311 ### Summary and Changelog The Kafka Connect transaction coordinator no longer stalls when the topic partition lookup fails while it starts a commit. - `ConnectTransactionCoordinator.startNewCommit()`: the `getLatestNumPartitions` lookup now runs inside the `try`, so a failed lookup takes the existing path that logs and reschedules START_COMMIT after `RESTART_COMMIT_DELAY_MS`, exactly like a failed `startCommit()`. Before, the `HoodieException` escaped to the event loop's catch-all, and no further commit was started until the coordinator was recreated by a task restart or a rebalance. - `TestConnectTransactionCoordinator`: a partition lookup that fails once no longer stops the coordinator from publishing START_COMMIT. ### Impact Liveness fix for the Kafka Connect sink: a single failed lookup, for example while the brokers are briefly unreachable as a commit starts, left every partition paused, so the connector stopped ingesting while its task stayed RUNNING. There are no public API, configuration, or storage format changes. ### Risk Level low. The change moves one statement into an existing `try`, ahead of `startCommit()`, so a failed lookup also leaves no instant behind. The new test fails without the fix (`Wanted but not invoked: kafkaControlAgent.publishMessage(...)`) and also when the lookup is moved after `startCommit()` (the first published commit time becomes `102`), and all 23 hudi-kafka-connect unit tests pass. I audited every path that schedules START_COMMIT, where exactly one is ever pending and every other exception path in the event loop still has a pending event that recovers it, and the shutdown path, where a lookup interrupted by `stop()` reaches the same catch and `submitEvent` schedules nothing once `hasStarted` is false. A lookup that keeps failing is now retried every 500 ms, like a failing `startCommit()`; each attempt creates a Kafka `AdminClient`, which #20303 closes, so this is best merged after #20303. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
