Re: delete topic causing spikes in fetch/metadata requests

[Zakee]

These errors are possibly caused by delete topic (esp. using v0.8.2.x), refer 
below issues.

https://issues.apache.org/jira/browse/KAFKA-2231
https://issues.apache.org/jira/browse/KAFKA-1194

The replica fetchers still querying their leaders for partitions of deleted 
topic. May need a restart of all brokers and possibly ZK (cleanup of data 
directory) to get rid of these errors. I had the same issue (using 0.10.1.0) 
which claims the issue above is fixed. However, rolling restart of all Zk 
instances only helped get rid of the error in my case.

Thanks.

> On Oct 16, 2016, at 8:37 AM, sunil kalva  wrote:
> 
> Hi
> Can you guys help me with this issue
> 
> On Oct 12, 2016 10:35 PM, "sunil kalva"  wrote:
> 
>> 
>> We are using kafka 0.8.2.2 (client and server), when ever we delete a
>> topic we see lot of errors in broker logs like below, and there is also a
>> spike in fetch/metadata requests. Can i correlate these errors with topic
>> delete or its a known issue. Since there is spike in metadata requests and
>> fetch requests broker throughput has comedown.
>> 
>> 
>> 
>> 
>> --
>> [2016-10-12 16:04:55,054] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,056] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,057] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,059] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,060] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,062] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,064] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,065] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,067] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,068] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 to
>> 202816546. (kafka.server.ReplicaManager)
>> [2016-10-12 16:04:55,070] ERROR [Replica Manager on Broker 4]: Error when
>> processing fetch request for partition [xyz,0] offset 161946645 from
>> consumer with correlation id 0. Possible cause: Request for offset
>> 161946645 but we only have log segments in the range 185487049 

Re: delete topic causing spikes in fetch/metadata requests

[sunil kalva]

Hi
Can you guys help me with this issue

On Oct 12, 2016 10:35 PM, "sunil kalva"  wrote:

>
> We are using kafka 0.8.2.2 (client and server), when ever we delete a
> topic we see lot of errors in broker logs like below, and there is also a
> spike in fetch/metadata requests. Can i correlate these errors with topic
> delete or its a known issue. Since there is spike in metadata requests and
> fetch requests broker throughput has comedown.
>
> 
> 
> 
> --
> [2016-10-12 16:04:55,054] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,056] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,057] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,059] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,060] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,062] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,064] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,065] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,067] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,068] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,070] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,072] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 16:04:55,073] ERROR [Replica Manager on Broker 4]: Error when
> processing fetch request for partition [xyz,0] offset 161946645 from
> consumer with correlation id 0. Possible cause: Request for offset
> 161946645 but we only have log segments in the range 185487049 to
> 202816546. (kafka.server.ReplicaManager)
> [2016-10-12 

delete topic causing spikes in fetch/metadata requests

[sunil kalva]

We are using kafka 0.8.2.2 (client and server), when ever we delete a topic
we see lot of errors in broker logs like below, and there is also a spike
in fetch/metadata requests. Can i correlate these errors with topic delete
or its a known issue. Since there is spike in metadata requests and fetch
requests broker throughput has comedown.

--
[2016-10-12 16:04:55,054] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,056] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,057] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,059] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,060] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,062] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,064] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,065] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,067] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,068] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,070] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,072] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,073] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to
202816546. (kafka.server.ReplicaManager)
[2016-10-12 16:04:55,075] ERROR [Replica Manager on Broker 4]: Error when
processing fetch request for partition [xyz,0] offset 161946645 from
consumer with correlation id 0. Possible cause: Request for offset
161946645 but we only have log segments in the range 185487049 to