Sean Quah created KAFKA-18905:
---------------------------------

             Summary: Avoid out of order sequence errors from multiple 
in-flight batches
                 Key: KAFKA-18905
                 URL: https://issues.apache.org/jira/browse/KAFKA-18905
             Project: Kafka
          Issue Type: Bug
          Components: producer 
            Reporter: Sean Quah


Consider a similar setup to KAFKA-9199:

The broker attempts to cache the state of the last 5 batches in order to enable 
duplicate detection. It is possible in some cases for this to result in a 
sequence such as the following:
 # Send batch n
 # Batch n successfully written, but response is delayed
 # Send Batch n+1, receive successful response for n+1
 # Send Batch n+2, receive successful response for n+2
 # Send Batch n+3, receive successful response for n+3
 # Send Batch n+4, receive successful response for n+4
 # Send Batch n+5, receive successful response for n+5
 # Batch n times out, or we receive a {{NOT_LEADER_OR_FOLLOWER}} response
 # Retry batch n, the response is {{OUT_OF_ORDER_SEQUENCE}}, retry again and 
again

This situation could happen with a {{max.in.flight.requests.per.connection}} as 
low as 2.

To avoid this situation, we can avoid putting batch n+5 in flight while batch n 
is in flight.




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to