suzhiking opened a new pull request, #23204: URL: https://github.com/apache/kafka/pull/23204
Under transactions V2, when the transaction coordinator aborts an open transaction on its own (e.g. after `transaction.timeout.ms` elapses), the abort bumps the producer epoch. A commit that was already in flight when the abort completed arrives with the pre-abort epoch, which matches the V2 EndTxn retry condition (stored epoch == request epoch + 1). It therefore passes the epoch check and fails the state check instead: the coordinator logs "state is COMPLETE_ABORT, but received transaction marker result to send: COMMIT" and returns `INVALID_TXN_STATE`, which producer clients treat as unconditionally fatal. Kafka Streams cannot recover from it and the affected StreamThread dies, even though the transaction was fully rolled back server-side and the commit never took effect. Under transactions V1 the same race fails the strict epoch check and returns `PRODUCER_FENCED`, which applications such as Kafka Streams handle gracefully by rebalancing; the fatal outcome is V2-only. This change returns the recoverable `INVALID_PRODUCER_EPOCH` instead of `INVALID_TXN_STATE` when a commit arrives in `COMPLETE_ABORT` at a retry epoch (both the epoch-bump and the producer-id-overflow variants). Only a coordinator-initiated abort can produce this combination: a client retrying its own EndTxn always carries the operation it originally sent, so a COMMIT at the pre-abort epoch means the producer never requested the abort. Producer clients already handle `INVALID_PRODUCER_EPOCH` on the EndTxn response (treated as `ProducerFencedException`), which Kafka Streams recovers from by rebalancing — restoring the V1 behavior for this race. A commit in `COMPLETE_ABORT` at the current epoch still returns `INVALID_TXN_STATE`, since that indicates a client-side bug rather than this race. Only transactions-V2 clients reach this code path, so no protocol or version-gating changes are needed. This is the EndTxn-path analogue of KAFKA-19690 (#20534), which made the same correction on the Produce path for producers that were mid-produce when the timeout abort landed. ### Testing - New `TransactionCoordinatorTest` cases: `COMPLETE_ABORT` + COMMIT at the pre-abort epoch returns `INVALID_PRODUCER_EPOCH` without appending to the transaction log, plus the epoch-overflow variant. - Updated `shouldReturnInvalidTxnRequestOnEndTxnRequestWhenStatusIsCompleteAbortAndResultIsNotAbort` to pin the current-epoch commit to `INVALID_TXN_STATE` for both TV0 and TV2 (it previously sent the retry epoch for TV2). - `./gradlew :core:test` for `TransactionCoordinatorTest` and `TransactionCoordinatorConcurrencyTest` passes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
