[
https://issues.apache.org/jira/browse/KAFKA-3129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vahid Hashemian updated KAFKA-3129:
-----------------------------------
Description:
I have been running a simple test case in which I have a text file
{{messages.txt}} with 1,000,000 lines (lines contain numbers from 1 to
1,000,000 in ascending order). I run the console consumer like this:
{{$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test}}
Topic {{test}} is on 1 partition with a replication factor of 1.
Then I run the console producer like this:
{{$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test <
messages.txt}}
Then the console starts receiving the messages. And about half the times it
goes all the way to 1,000,000. But, in other cases, it stops short, usually at
999,735.
I tried running another console consumer on another machine and both consumers
behave the same way. I can't see anything related to this in the logs.
I also ran the same experiment with a similar file of 10,000 lines, and am
getting a similar behavior. When the consumer does not receive all the 10,000
messages it usually stops at 9,864.
was:
I have been running a simple test case in which I have a text file
{{messages.txt}} with 1,000,000 lines (lines contain numbers from 1 to
1,000,000 in ascending order). I run the console consumer like this:
{{$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test}}
Topic {{test}} is on 1 partition with a replication factor of 1.
Then I run the console producer like this:
{{$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test <
messages.txt}}
Then the console starts receiving the messages. And about half the times it
goes all the way to 1,000,000. But, in other cases, it stops short, usually at
999,735.
I tried running another console consumer on another machine and both consumers
behave the same way. I can't see anything related to this in the logs.
> Potential Console Producer/Consumer Issue
> -----------------------------------------
>
> Key: KAFKA-3129
> URL: https://issues.apache.org/jira/browse/KAFKA-3129
> Project: Kafka
> Issue Type: Bug
> Components: consumer, producer
> Affects Versions: 0.9.0.0
> Reporter: Vahid Hashemian
> Assignee: Neha Narkhede
>
> I have been running a simple test case in which I have a text file
> {{messages.txt}} with 1,000,000 lines (lines contain numbers from 1 to
> 1,000,000 in ascending order). I run the console consumer like this:
> {{$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test}}
> Topic {{test}} is on 1 partition with a replication factor of 1.
> Then I run the console producer like this:
> {{$ bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test <
> messages.txt}}
> Then the console starts receiving the messages. And about half the times it
> goes all the way to 1,000,000. But, in other cases, it stops short, usually
> at 999,735.
> I tried running another console consumer on another machine and both
> consumers behave the same way. I can't see anything related to this in the
> logs.
> I also ran the same experiment with a similar file of 10,000 lines, and am
> getting a similar behavior. When the consumer does not receive all the 10,000
> messages it usually stops at 9,864.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
