[
https://issues.apache.org/jira/browse/KAFKA-5766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16148910#comment-16148910
]
Sebastian Bernauer commented on KAFKA-5766:
-------------------------------------------
Thanks for your reply.
What loop are you referring to?
The behavior is just as expected, beside the fact that the cpu usage is at 100%.
If we add this to the configuration:
{code:java}
configs.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, 1000);
configs.put(ConsumerConfig.RECONNECT_BACKOFF_MS_CONFIG, 1000);
{code}
the log still rapidly shows the messages
{noformat}
14:13:51.878 [main] DEBUG
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient - Cancelled
FETCH request ClientRequest(expectResponse=true,
callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@5e4544f2,
request=RequestSend(header={api_key=1,api_version=3,correlation_id=530041,client_id=consumer-1},
body={replica_id=-1,max_wait_time=500,min_bytes=1,max_bytes=52428800,topics=[{topic=foo,partitions=[{partition=0,fetch_offset=0,max_bytes=1048576}]}]}),
createdTimeMs=1504181631878, sendTimeMs=0) with correlation id 530041 due to
node 0 being disconnected
14:13:51.878 [main] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher -
Fetch request to XXXXXXXXXX:9092 (id: 0 rack: null) failed
org.apache.kafka.common.errors.DisconnectException: null
14:13:51.878 [main] DEBUG
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient - Cancelled
FETCH request ClientRequest(expectResponse=true,
callback=org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler@200a0ba7,
request=RequestSend(header={api_key=1,api_version=3,correlation_id=530042,client_id=consumer-1},
body={replica_id=-1,max_wait_time=500,min_bytes=1,max_bytes=52428800,topics=[{topic=foo,partitions=[{partition=0,fetch_offset=0,max_bytes=1048576}]}]}),
createdTimeMs=1504181631878, sendTimeMs=0) with correlation id 530042 due to
node 0 being disconnected
14:13:51.878 [main] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher -
Fetch request to XXXXXXXXXX:9092 (id: 0 rack: null) failed
org.apache.kafka.common.errors.DisconnectException: null
and so on...
{noformat}
and the usage is still at ~ 100%.
Why isn't there a delay of about a second before it tries a reconnect / poll
once again?
> Very high CPU-load of consumer when broker is down
> --------------------------------------------------
>
> Key: KAFKA-5766
> URL: https://issues.apache.org/jira/browse/KAFKA-5766
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Reporter: Sebastian Bernauer
>
> Hi,
> i have a single broker instance at localhost.
> I set up a Consumer with the following code:
> {code:java}
> Map<String, Object> configs = new HashMap<>();
> configs.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,
> "localhost:9092");
> configs.put(ConsumerConfig.GROUP_ID_CONFIG, "gh399");
> configs.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG,
> StringDeserializer.class);
> configs.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG,
> StringDeserializer.class);
> KafkaConsumer<String, String> consumer = new KafkaConsumer<>(configs);
> consumer.assign(Collections.singletonList(new TopicPartition("foo",
> 0)));
> while (true) {
> ConsumerRecords<String, String> records = consumer.poll(1000);
> System.out.println(records.count());
> }
> {code}
> This works all fine, until i shut down the broker.
> If i do so, it causes a 100% CPU-load of my application.
> After starting the broker again the usage decreases back to a normal level.
> It would be very nice if you could help me!
> Thanks,
> Sebastian
> Spring-Kafka: 2.0.0.M3
> Kafka: 0.10.2.0
> JDK: 1.8.0_121
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
