Hi all, I'd like to raise a question about the IllegalStateException thrown by KafkaProducer.throwIfProducerClosed().
The problem KafkaProducer currently throws IllegalStateException from two unrelated code paths: - throwIfProducerClosed() — producer was closed (an operational event) - throwIfInPreparedState() — transactional 2PC state violation (a programming error) There's no way to programmatically distinguish these without matching on the exception message, which isn't part of the public API contract and has already changed across versions (e.g. "Cannot send after the producer is closed" vs "Cannot perform operation after producer has been closed"). This matters for applications that need to recover from a closed producer (e.g. during MSK security patch update) by detecting the failure and recreating the producer instance. Prior discussion within the Kafka project This isn't a new observation. In PR #5027 <https://github.com/apache/kafka/pull/5027> (KAFKA-6897, 2018), committer @cmccabe noted: *"I'm not sure that IllegalStateException is the right one to throw, since the state is legal and reachable."* The PR author agreed and suggested changing these to a more suitable exception in a follow-up PR, and committer @hachikuji commented: *"We should try and distinguish legitimate illegal state errors."* That follow-up never happened. Additionally, KIP-691 <https://cwiki.apache.org/confluence/display/KAFKA/KIP-691%3A+Enhance+Transactional+Producer+Exception+Handling> classified IllegalStateException as indicating *"a bug",* but a closed producer is a legitimate operational condition, not a bug. Impact across the ecosystem - Kafka's own old Scala producer (0.8.x) had a dedicated ProducerClosedException,it was lost in the Java rewrite I guess - Apache Samza (KafkaSystemProducer) references ProducerClosedException by name in its comments and has to work around its absence - Multiple StackOverflow threads show developers resorting to string matching or stack trace inspection Possible solution Would it be possible to introduce a dedicated exception for this, similar to how Kafka already has ProducerFencedException, OutOfOrderSequenceException, etc.? Would there be interest in a KIP for this? Happy to write one up. Thanks, Daulet
