Yiquan Zhou created KAFKA-4348:
----------------------------------
Summary: On Mac OS, KafkaConsumer.poll returns 0 when there are
still messages on Kafka server
Key: KAFKA-4348
URL: https://issues.apache.org/jira/browse/KAFKA-4348
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 0.10.0.1, 0.9.0.1, 0.9.0.0
Environment: Max OS X EI Capitan, Java 1.8.0_77
Reporter: Yiquan Zhou
Steps to reproduce:
1. start the zookeeper and kafka server using the default properties from the
distribution:
$ bin/zookeeper-server-start.sh config/zookeeper.properties
$ bin/kafka-server-start.sh config/server.properties
2. create a Kafka consumer using the Java API KafkaConsumer.poll(long timeout).
It polls the records from the server every second (timeout set to 1000) and
prints the number of records polled. The code can be found here:
https://gist.github.com/yiquanzhou/a94569a2c4ec8992444c83f3c393f596
3. use bin/kafka-verifiable-producer.sh to generate some messages:
$ bin/kafka-verifiable-producer.sh --topic connect-test --max-messages 200000
--broker-list localhost:9092
wait until all 200k messages are generated and sent to the server.
4. Run the consumer Java code. In the output console of the consumer, we can
see that the consumer starts to poll some records, then it polls 0 records for
several seconds before polling some more. like this:
polled 27160 records
polled 0 records
polled 0 records
polled 0 records
polled 0 records
polled 0 records
polled 26886 records
polled 26886 records
polled 0 records
polled 0 records
polled 0 records
polled 0 records
polled 0 records
polled 26701 records
polled 26214 records
The impact of this bug is that it slows down the consumption of messages a lot.
And if an eventListener is attached to the idle event, it could be triggered
even if there are still messages on the server.
It is only reproducible on Mac OS X but neither on Linux nor Windows.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
