Sean Quah created KAFKA-18905: --------------------------------- Summary: Avoid out of order sequence errors from multiple in-flight batches Key: KAFKA-18905 URL: https://issues.apache.org/jira/browse/KAFKA-18905 Project: Kafka Issue Type: Bug Components: producer Reporter: Sean Quah
Consider a similar setup to KAFKA-9199: The broker attempts to cache the state of the last 5 batches in order to enable duplicate detection. It is possible in some cases for this to result in a sequence such as the following: # Send batch n # Batch n successfully written, but response is delayed # Send Batch n+1, receive successful response for n+1 # Send Batch n+2, receive successful response for n+2 # Send Batch n+3, receive successful response for n+3 # Send Batch n+4, receive successful response for n+4 # Send Batch n+5, receive successful response for n+5 # Batch n times out, or we receive a {{NOT_LEADER_OR_FOLLOWER}} response # Retry batch n, the response is {{OUT_OF_ORDER_SEQUENCE}}, retry again and again This situation could happen with a {{max.in.flight.requests.per.connection}} as low as 2. To avoid this situation, we can avoid putting batch n+5 in flight while batch n is in flight. -- This message was sent by Atlassian Jira (v8.20.10#820010)