[
https://issues.apache.org/jira/browse/KAFKA-4739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vipul Singh updated KAFKA-4739:
-------------------------------
Description:
We are seeing an issue with our kafka consumer where it seems to go into an
infinite loop while polling, trying to fetch data from kafka. We are seeing the
heartbeat requests on the broker from the consumer, but nothing else from the
kafka consumer.
We enabled debug level logging on the consumer, and see these logs:
DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient:
Cancelled FETCH request ClientRequest(metadata info) with correlation id abc
due to node xyz being disconnected
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.common.errors.DisconnectException: null
DEBUG org.apache.kafka.clients.NetworkClient: Initiating connection to node abc
at nodename:port
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUGorg.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.NetworkClient: Completed connection to node xyz
DEBUG org.apache.kafka.clients.Metadata: Updated cluster metadata version 4 to
Cluster(cluster_info)
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
And this just goes on. The way we have been able to replicate this issue, is by
restarting the process in multiple successions.
was:
We are seeing an issue with our kafka consumer where it seems to go into an
infinite loop while polling, trying to fetch data from kafka. We are seeing the
heartbeat requests on the broker from the consumer, but nothing else from the
kafka consumer.
We enabled debug level logging on the consumer, and see these logs:
DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient:
Cancelled FETCH request ClientRequest(metadata info) with correlation id abc
due to node xyz being disconnected
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.common.errors.DisconnectException: null
DEBUG org.apache.kafka.clients.NetworkClient: Initiating connection to node abc
at nodename:port
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUGorg.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
! org.apache.kafka.clients.consumer.internals.SendFailedException: null
DEBUG org.apache.kafka.clients.NetworkClient: Completed connection to node xyz
DEBUG org.apache.kafka.clients.Metadata: Updated cluster metadata version 4 to
Cluster(cluster_info)
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator: Received
successful heartbeat response.
And this just goes. The way we have been able to replicate this issue, is by
restarting the process in multiple successions.
> KafkaConsumer poll going into an infinite loop
> ----------------------------------------------
>
> Key: KAFKA-4739
> URL: https://issues.apache.org/jira/browse/KAFKA-4739
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.9.0.1
> Reporter: Vipul Singh
>
> We are seeing an issue with our kafka consumer where it seems to go into an
> infinite loop while polling, trying to fetch data from kafka. We are seeing
> the heartbeat requests on the broker from the consumer, but nothing else from
> the kafka consumer.
> We enabled debug level logging on the consumer, and see these logs:
> DEBUG org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient:
> Cancelled FETCH request ClientRequest(metadata info) with correlation id abc
> due to node xyz being disconnected
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.common.errors.DisconnectException: null
> DEBUG org.apache.kafka.clients.NetworkClient: Initiating connection to node
> abc at nodename:port
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUGorg.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.consumer.internals.Fetcher: Fetch failed
> ! org.apache.kafka.clients.consumer.internals.SendFailedException: null
> DEBUG org.apache.kafka.clients.NetworkClient: Completed connection to node xyz
> DEBUG org.apache.kafka.clients.Metadata: Updated cluster metadata version 4
> to Cluster(cluster_info)
> DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator:
> Received successful heartbeat response.
> DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator:
> Received successful heartbeat response.
> DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator:
> Received successful heartbeat response.
> DEBUG org.apache.kafka.clients.consumer.internals.AbstractCoordinator:
> Received successful heartbeat response.
> And this just goes on. The way we have been able to replicate this issue, is
> by restarting the process in multiple successions.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
