[
https://issues.apache.org/jira/browse/KAFKA-12776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347259#comment-17347259
]
A. Sophie Blee-Goldman commented on KAFKA-12776:
------------------------------------------------
[~neeraj.vaidya] although the ongoing
[KIP-739|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=181306446]
isn't actually trying to solve the same problem, I expect there might be
considerable overlap in the implementation and design considerations between
that KIP and what you're trying to accomplish. I recommend starting out by
reading up on that KIP to see how the current design aligns with your thinking.
Admittedly I haven't been following it closely, but my impression is that the
proposal involves introducing another queue that sits outside the regular
partitioned buffers. It sounds like this approach could be a good fit for
solving the re-ordering problem as well.
If you agree, you can follow up by sending a message to the KIP-739 discussion
thread and outlining how their design could apply to your case, and whether
that's something that could be worked into this KIP and its general design. I
suspect it would not require much additional work or expand the current scope
too much, but if it does maybe you can offer to help out with this aspect of
the KIP.
> Producer sends messages out-of-order inspite of enabling idempotence
> --------------------------------------------------------------------
>
> Key: KAFKA-12776
> URL: https://issues.apache.org/jira/browse/KAFKA-12776
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 2.6.0, 2.7.0
> Environment: Linux RHEL 7.9 and Ubuntu 20.04
> Reporter: NEERAJ VAIDYA
> Priority: Major
> Attachments: mocker.zip
>
>
> I have an Apache Kafka 2.6 Producer which writes to topic-A (TA).
> My application is basically a Spring boot web-application which accepts JSON
> payloads via HTTP and then pushes each to a Kafka topic. I also use Spring
> Cloud Stream Kafka in the application to create and use a Producer.
> For one of my failure handling test cases, I shutdown the Kafka cluster while
> my applications are running. (Note : No messages have been published to the
> Kafka cluster before I stop the cluster)
> When the producer application tries to write messages to TA, it cannot
> because the cluster is down and hence (I assume) buffers the messages. Let's
> say it receives 4 messages m1,m2,m3,m4 in increasing time order. (i.e. m1 is
> first and m4 is last).
> When I bring the Kafka cluster back online, the producer sends the buffered
> messages to the topic, but they are not in order. I receive for example, m2
> then m3 then m1 and then m4.
> Why is that ? Is it because the buffering in the producer is multi-threaded
> with each producing to the topic at the same time ?
> My project code is attached herewith.
> I can confirm that I have enabled idempotence. I have also tried with
> ```max.in.flight.requests=1```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
