[jira] [Commented] (KAFKA-8021) KafkaProducer.flush() can show unexpected behavior when a batch is split
[ https://issues.apache.org/jira/browse/KAFKA-8021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781850#comment-16781850 ] Abhishek Mendhekar commented on KAFKA-8021: --- [~sriharsha] the issue is in 0.11 and higher version (including latest trunk). Since this is a very small race I am trying to reproduce this in an integration test but I was able to verify it via unit tests. > KafkaProducer.flush() can show unexpected behavior when a batch is split > > > Key: KAFKA-8021 > URL: https://issues.apache.org/jira/browse/KAFKA-8021 > Project: Kafka > Issue Type: Bug > Components: clients, producer >Affects Versions: 0.11.0.0 >Reporter: Abhishek Mendhekar >Assignee: Abhishek Mendhekar >Priority: Major > > KafkaProducer.flush() marks the flush in progress and then waits for all > incomplete batches to be completed (waits on the producer batch futures to > finish). > The behavior is seen when a batch is split due to MESSAGE_TOO_LARGE exception. > The large batch is split into smaller batches (2 or more) but > ProducerBatch.split() marks the large batch future as complete before adding > the new batches in the incomplete list of batches. At this time if the > KafkaProducer.flush() is called then it'll make a copy of existing incomplete > list of batches and waits for them to complete while ignoring the large batch > that was split into smaller batches. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-8021) KafkaProducer.flush() can show unexpected behavior when a batch is split
[ https://issues.apache.org/jira/browse/KAFKA-8021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781820#comment-16781820 ] Sriharsha Chintalapani commented on KAFKA-8021: --- [~amendhekar] have you noticed this issue with the latest release of 2.1.1 > KafkaProducer.flush() can show unexpected behavior when a batch is split > > > Key: KAFKA-8021 > URL: https://issues.apache.org/jira/browse/KAFKA-8021 > Project: Kafka > Issue Type: Bug > Components: clients, producer >Affects Versions: 0.11.0.0 >Reporter: Abhishek Mendhekar >Assignee: Abhishek Mendhekar >Priority: Major > > KafkaProducer.flush() marks the flush in progress and then waits for all > incomplete batches to be completed (waits on the producer batch futures to > finish). > The behavior is seen when a batch is split due to MESSAGE_TOO_LARGE exception. > The large batch is split into smaller batches (2 or more) but > ProducerBatch.split() marks the large batch future as complete before adding > the new batches in the incomplete list of batches. At this time if the > KafkaProducer.flush() is called then it'll make a copy of existing incomplete > list of batches and waits for them to complete while ignoring the large batch > that was split into smaller batches. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
