krishvishal opened a new issue, #4522: URL: https://github.com/apache/iggy/issues/4522
Follow-up to #3545. ## Summary There is no transaction coordinator. FindCoordinator for a transaction answers 53, and InitProducerId with a transactional id answers 35. AddPartitionsToTxn (24), AddOffsetsToTxn (25), EndTxn (26) and TxnOffsetCommit (28) are not served (`gateways/kafka/docs/SCOPE.md`, "Transactions"). Who hits it: - ksqlDB writes every statement in a transaction, so it cannot start. - Flink `DeliveryGuarantee.EXACTLY_ONCE`, Kafka Streams `exactly_once_v2`, Spring `transactionIdPrefix` and Connect `exactly.once.source.support` fail at startup. - A Connect leader with exactly-once sources retries every 250 ms forever and serves no requests. - The Java error is `TransactionalIdAuthorizationException`, which sends users to their ACLs. ## Open question `gateways/kafka/docs/SCOPE.md` says transactions are not planned. Full client compatibility needs that decision revisited. This issue tracks the decision. If the decision changes, it also tracks the work. ## Scope - [ ] A transaction coordinator with replicated state - [ ] Control batches and abort markers in Iggy - [ ] Fetch with the last stable offset and the aborted transaction list, so `read_committed` works - [ ] TxnOffsetCommit into the `kafka.cg.` offsets - [ ] Crash and partition tests ## Acceptance criteria - [ ] Kafka Streams with `exactly_once_v2` runs through the gateway, and a `read_committed` consumer never sees an aborted record - [ ] ksqlDB starts ## References - Connect leader loop: [DistributedHerder.java:440-477](https://github.com/apache/kafka/blob/b15258c7398889248d7fbf2a2218cc01346ebf92/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L440-L477) - ksqlDB: [KsqlInternalTopicUtils.java](https://github.com/confluentinc/ksql/blob/b665cf557f9bd29985cc52dc087271f256d36e1d/ksqldb-rest-app/src/main/java/io/confluent/ksql/rest/util/KsqlInternalTopicUtils.java) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
