[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
I am observing log filling up on kafka_2.10-0.8.2.0.
Following are the logs on one of kafka node:
On Topic Creation:
[2015-07-07 20:28:17,579] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [tempv1,0] (kafka.server.ReplicaFetcherManager)
[2015-07-07 20:28:17,582] INFO Completed load of log tempv1-0 with log end
offset 0 (kafka.log.Log)
[2015-07-07 20:28:17,583] INFO Created log for partition [tempv1,0] in
/tmp/kafka-logs-server1 with properties {segment.index.bytes -> 10485760,
file.delete.delay.ms -> 6, segment.bytes -> 536870912, flush.ms ->
9223372036854775807, delete.retention.ms -> 8640, index.interval.bytes
-> 4096, retention.bytes -> -1, min.insync.replicas -> 1, cleanup.policy ->
delete, unclean.leader.election.enable -> true, segment.ms -> 60480,
max.message.bytes -> 112, flush.messages -> 9223372036854775807,
min.cleanable.dirty.ratio -> 0.5, retention.ms -> 60480,
segment.jitter.ms -> 0}. (kafka.log.LogManager)
[2015-07-07 20:28:17,583] WARN Partition [tempv1,0] on broker 0: No
checkpointed highwatermark is found for partition [tempv1,0]
(kafka.cluster.Partition)
[2015-07-07 20:28:18,388] ERROR [Replica Manager on Broker 0]: Error when
processing fetch request for partition [tempv1,0] offset -1 from consumer
with correlation id 0. Possible cause: Request for offset -1 but we only
have log segments in the range 0 to 0. (kafka.server.ReplicaManager)
On Topic Deletion:
[2015-07-07 20:29:05,075] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [tempv1,0] (kafka.server.ReplicaFetcherManager)
[2015-07-07 20:29:05,076] INFO [ReplicaFetcherManager on broker 0] Removed
fetcher for partitions [tempv1,0] (kafka.server.ReplicaFetcherManager)
[2015-07-07 20:29:05,076] WARN [Replica Manager on Broker 0]: Fetch request
with correlation id 0 from client Channel_tempv1_0 on partition [tempv1,0]
failed due to Partition [tempv1,0] doesn't exist on 0
(kafka.server.ReplicaManager)
[2015-07-07 20:29:05,077] INFO Deleting index
/tmp/kafka-logs-server1/tempv1-0/.index
(kafka.log.OffsetIndex)
[2015-07-07 20:29:05,077] INFO Deleted log for partition [tempv1,0] in
/tmp/kafka-logs-server1/tempv1-0. (kafka.log.LogManager)
[2015-07-07 20:29:05,095] INFO Closing socket connection to /10.240.213.24.
(kafka.network.Processor)
[2015-07-07 20:29:05,095] WARN [Replica Manager on Broker 0]: Fetch request
with correlation id 0 from client Channel_tempv1_0 on partition [tempv1,0]
failed due to Partition [tempv1,0] doesn't exist on 0
(kafka.server.ReplicaManager)
[2015-07-07 20:29:05,096] INFO Closing socket connection to /10.240.213.24.
(kafka.network.Processor)
[2015-07-07 20:29:05,097] WARN [Replica Manager on Broker 0]: Fetch request
with correlation id 0 from client Channel_tempv1_0 on partition [tempv1,0]
failed due to Partition [tempv1,0] doesn't exist on 0
(kafka.server.ReplicaManager)
[2015-07-07 20:29:05,098] INFO Closing socket connection to /10.240.213.24.
(kafka.network.Processor)
..
..
At the time of deletion, there was no java client running and accessing
kafka topic.
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553460#comment-14553460
]
Greg Lloyd commented on KAFKA-1479:
---
I am not able to locate the directories that Manasi referenced, anybody mind
pointing out where they are located. I have kafka isntalled in /usr/local/kafka
and zookeeper installed in /usr/local/zookeeper.
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249071#comment-14249071
]
Arup Malakar commented on KAFKA-1479:
-
For people who may stumble upon this JIRA, the steps mentioned by [~manasi] in
https://issues.apache.org/jira/browse/KAFKA-1479?focusedCommentId=14017044&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14017044
worked for me as well.
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017264#comment-14017264
]
Manasi Manasi commented on KAFKA-1479:
--
Ok, resolved the issue. Basically deleted data files and zookeeper data for the
'deleted' topics:
/brokers/topics/my_topic
/consumers/[groupId]/owners/my_topic
/consumers/[groupId]/offsets/my_topic
and bounced the cluster.
Looks like that solved the problem.
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>Assignee: Jay Kreps
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017044#comment-14017044
]
Manasi Manasi commented on KAFKA-1479:
--
Nope. That didn't solve the problem. I still see the log lines. Any way for me
to hard delete the topics that have already been deleted using the kafkak
delete topic tool, without a downtime for the cluster?
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>Assignee: Jay Kreps
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016184#comment-14016184
]
Jun Rao commented on KAFKA-1479:
I am not sure. You will have to give it a try.
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>Assignee: Jay Kreps
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14016065#comment-14016065
]
Manasi Manasi commented on KAFKA-1479:
--
Thanks for the reply, Jun. I will upgrade to 0.8.1.1.
But why would this manifest all of a sudden? Most of the topics have been
deleted a while back and I didn't see any logging issues. Also, if I switch to
0.8.1.1, is the check for deleted topics disabled? So that I won't see the log
lines anymore.
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>Assignee: Jay Kreps
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (KAFKA-1479) Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition info for deleted topics
[
https://issues.apache.org/jira/browse/KAFKA-1479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14015451#comment-14015451
]
Jun Rao commented on KAFKA-1479:
Delete topics doesn't quite work in 0.8.1. So we took it out in 0.8.1.1 and
plan to fix it in 0.8.2.
> Logs filling up while Kafka ReplicaFetcherThread tries to retrieve partition
> info for deleted topics
>
>
> Key: KAFKA-1479
> URL: https://issues.apache.org/jira/browse/KAFKA-1479
> Project: Kafka
> Issue Type: Bug
> Components: log
>Affects Versions: 0.8.1
> Environment: CentOS
>Reporter: Manasi Manasi
>Assignee: Jay Kreps
>
> Started noticing that logs are filling up fast with lines like this:
> {quote}
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,26]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,218] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-28,38]
> failed due to Topic sams_2014-05-28 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,20]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-22,46]
> failed due to Topic sams_2014-05-22 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:18:08,219] WARN [KafkaApi-2] Fetch request with correlation id
> 10049 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-27,8]
> failed due to Topic sams_2014-05-27 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> The above is from kafkaServer.out. Also seeing errors in server.log:
> {quote}
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,19] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-30,4]
> failed due to Topic sams_2014-05-30 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-24,34] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-26,41] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [2014-05-21,0] failed
> due to Topic 2014-05-21 either doesn't exist or is in the process of being
> deleted (kafka.server.KafkaApis)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-28,42] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] ERROR [ReplicaFetcherThread-0-0], Error for
> partition [sams_2014-05-22,21] to broker 0:class
> kafka.common.UnknownTopicOrPartitionException
> (kafka.server.ReplicaFetcherThread)
> [2014-06-01 15:23:52,788] WARN [KafkaApi-2] Fetch request with correlation id
> 10887 from client ReplicaFetcherThread-0-2 on partition [sams_2014-05-20,26]
> failed due to Topic sams_2014-05-20 either doesn't exist or is in the process
> of being deleted (kafka.server.KafkaApis)
> {quote}
> All these partitions belong to deleted topics. Nothing changed on our end
> when we started noticing these logs filling up. Any ideas what is going on?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
