[
https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17344184#comment-17344184
]
A. Sophie Blee-Goldman commented on KAFKA-12776:
------------------------------------------------
I think the messages should be ordered within a batch regardless of whether the
key is strictly the same or not, all that matters is that the partition is the
same. But yes, if you've set the max in-flight request to 1 then the idempotent
producer should guarantee message ordering, ie messages should be produced in
the order that you called #send. Just to be clear, this does not mean that
messages are guaranteed to be in timestamp order, as you can produce messages
with out-of-order timestamps.
It's always possible there is a bug or regression in the Producer, but we
should first rule out other things – for example can you just verify in the
logs that _max.in.flight.requests.per.connection_ __ is being set to 1, and
not overridden by Spring for some reason? Assuming this config is being set
correctly, can you confirm that the messages are indeed being sent in the
desired order, either by providing sample code or by logging the message in
your application before invoking Producer#send so that there's a record of
which messages were sent at what time?
> Producer sends messages out-of-order inspite of enabling idempotence
> --------------------------------------------------------------------
>
> Key: KAFKA-12776
> URL: https://issues.apache.org/jira/browse/KAFKA-12776
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 2.6.0, 2.7.0
> Environment: Linux RHEL 7.9 and Ubuntu 20.04
> Reporter: NEERAJ VAIDYA
> Priority: Major
> Attachments: mocker.zip
>
>
> I have an Apache Kafka 2.6 Producer which writes to topic-A (TA).
> My application is basically a Spring boot web-application which accepts JSON
> payloads via HTTP and then pushes each to a Kafka topic. I also use Spring
> Cloud Stream Kafka in the application to create and use a Producer.
> For one of my failure handling test cases, I shutdown the Kafka cluster while
> my applications are running. (Note : No messages have been published to the
> Kafka cluster before I stop the cluster)
> When the producer application tries to write messages to TA, it cannot
> because the cluster is down and hence (I assume) buffers the messages. Let's
> say it receives 4 messages m1,m2,m3,m4 in increasing time order. (i.e. m1 is
> first and m4 is last).
> When I bring the Kafka cluster back online, the producer sends the buffered
> messages to the topic, but they are not in order. I receive for example, m2
> then m3 then m1 and then m4.
> Why is that ? Is it because the buffering in the producer is multi-threaded
> with each producing to the topic at the same time ?
> My project code is attached herewith.
> I can confirm that I have enabled idempotence. I have also tried with
> ```max.in.flight.requests=1```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
