NEERAJ VAIDYA created KAFKA-12776:
-------------------------------------
Summary: Producer sends messages out-of-order inspite of enabling
idempotence
Key: KAFKA-12776
URL: https://issues.apache.org/jira/browse/KAFKA-12776
Project: Kafka
Issue Type: Bug
Components: producer
Affects Versions: 2.7.0, 2.6.0
Environment: Linux RHEL 7.9 and Ubuntu 20.04
Reporter: NEERAJ VAIDYA
I have an Apache Kafka 2.6 Producer which writes to topic-A (TA). I also have a
Kafka streams application which consumes from TA and writes to topic-B (TB). In
the streams application, I have a custom timestamp extractor which extracts the
timestamp from the message payload.
For one of my failure handling test cases, I shutdown the Kafka cluster while
my applications are running. (Note : No messages have been published to the
Kafka cluster before I stop the cluster)
When the producer application tries to write messages to TA, it cannot because
the cluster is down and hence (I assume) buffers the messages. Let's say it
receives 4 messages m1,m2,m3,m4 in increasing time order. (i.e. m1 is first and
m4 is last).
When I bring the Kafka cluster back online, the producer sends the buffered
messages to the topic, but they are not in order. I receive for example, m2
then m3 then m1 and then m4.
Why is that ? Is it because the buffering in the producer is multi-threaded
with each producing to the topic at the same time ?
My application is basically a Spring boot web-application which accepts JSON
payloads and then pushes them to a Kafka topic. I also use Spring Cloud Stream
Kafka within it to create and use a Producer.
My project code is attached herewith.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
