Savonitar opened a new pull request, #238: URL: https://github.com/apache/flink-connector-kafka/pull/238
## What is the purpose of the change A standalone CLI tool (`KafkaTransactionTool`) that allows operators to manually abort or commit lingering Kafka transactions left behind when a Flink job with exactly-once Kafka sink stops unexpectedly (crash, kill, cancellation) and is not restarted. Open transactions block downstream consumers using `read_committed` isolation (stuck LSO) and can lead to data loss - the data is checkpointed in Flink but never committed to Kafka, and will be lost when the transaction times out and is aborted by the broker. The tool provides two operations: - **Abort**: Fences the transactional producer via `initTransactions()`, causing the broker to abort the open transaction. - **Commit**: Resumes the transaction using the exact producerId and epoch (stored in Flink checkpoint state via `KafkaCommittable`) and commits it. ## Brief change log - Add `KafkaTransactionManager` - core logic for abort/commit operations - Add `KafkaTransactionTool` - CLI entry point using commons-cli - Package CLI as uber-jar via maven-shade-plugin ## Verifying this change This change includes integration tests (`KafkaTransactionManagerITCase`) that: - Create lingering transactions, abort them, and verify state - Create lingering transactions, commit them with correct producerId/epoch, and verify committed records are readable - Verify transaction handling during savepoint-based job lifecycle Unit tests (`KafkaTransactionManagerTest`, `KafkaTransactionToolTest`) cover input validation and CLI argument parsing. ## Does this pull request potentially affect one of the following parts - Dependencies (does it add or upgrade a dependency): yes (new submodule) - The public API: no - Anything that could affect deployment: no ## Documentation - Does this pull request introduce a new feature? yes - If yes, how is the feature documented? JavaDoc + CLI `--help` output -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
