Vimal Sharma created KAFKA-4164:
-----------------------------------
Summary: Kafka produces excessive logs when publishing message to
non-existent topic
Key: KAFKA-4164
URL: https://issues.apache.org/jira/browse/KAFKA-4164
Project: Kafka
Issue Type: Bug
Reporter: Vimal Sharma
When a message is published to a topic which is not already created(and
auto.create.topics.enable is set to false), Kafka produces excessive WARN logs
stating that metadata could not be fetched. Below are the logs
2016-08-22 06:43:47,655 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1177 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:47,756 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1178 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:47,858 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1179 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:47,961 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1180 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:48,062 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1181 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:48,165 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1182 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:48,265 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1183 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:48,366 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1184 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
2016-08-22 06:43:48,467 WARN [kafka-producer-network-thread | producer-1]:
clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
fetching metadata with correlation id 1185 :
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
The error is not communicated to the caller so if these logs are suppressed by
setting Kafka log level to ERROR, there is no way to debug the issue. It would
be helpful if the error message( for example
{ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}) can be communicated to the caller.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
