Sergey Alaev created KAFKA-4533:
-----------------------------------
Summary: subscribe() then poll() on new topic is very slow when
subscribed to many topics
Key: KAFKA-4533
URL: https://issues.apache.org/jira/browse/KAFKA-4533
Project: Kafka
Issue Type: Bug
Components: clients
Affects Versions: 0.10.1.0
Reporter: Sergey Alaev
Given following case:
consumer.subscribe(my_new_topic, [249 existing topics])
publisher.send(my_new_topic, key, value)
poll(10) until data from my_new_topic arrives
I see data from `my_new_topic` only after approx. 90 seconds.
If I subscribe only to my_new_topic, I get results within seconds.
Logs contain lots of lines like this:
19:28:07.972 [kafka-thread] DEBUG
org.apache.kafka.clients.consumer.internals.Fetcher - Resetting offset for
partition demo.com_recipient-2-0 to earliest offset.
19:28:08.247 [kafka-thread] DEBUG
org.apache.kafka.clients.consumer.internals.Fetcher - Fetched {timestamp=-1,
offset=0} for partition demo.com_recipient-2-0
Probably you should do that in batch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
