kafka (at least out of the box as it is now) is not an exactly-once system. its an at-least-once system, meaning the scenario you described (and similar ones involving socket disconnections, for example) exist by design.
there is a KIP for adding exactly once guarantees (among other things) that you can read here - https://cwiki.apache.org/confluence/display/KAFKA/KIP-98+-+Exactly+Once+Delivery+and+Transactional+Messaging On Fri, Mar 31, 2017 at 4:07 AM, Yang Cui <y...@freewheel.tv> wrote: > Hi All, > There is a JIRA issue which I submitted: https://issues.apache.org/ > jira/browse/KAFKA-4951; > It is about the Sending duplicated message by KafkaProducer, Could you > please check it? > Thanks. > > Yang Cui > >
