[
https://issues.apache.org/jira/browse/ATLAS-1131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15430833#comment-15430833
]
Vimal Sharma commented on ATLAS-1131:
-------------------------------------
Property max.block.ms is used by Kafka producer to keep retrying for topic
metadata in case of failure. The default is 60000 ms. Setting it to 1000 ms to
reduce the flood of WARN logs.
> Atlas hooks produce excess Kafka logs if Kafka topic is not created already
> ---------------------------------------------------------------------------
>
> Key: ATLAS-1131
> URL: https://issues.apache.org/jira/browse/ATLAS-1131
> Project: Atlas
> Issue Type: Bug
> Reporter: Vimal Sharma
> Assignee: Vimal Sharma
> Attachments: ATLAS-1131.patch
>
>
> Hooks for Atlas publish messages to a Kafka topic named ATLAS_HOOK. If this
> topic is not present and Atlas does not have permission to create this topic,
> Kafka produces excessive logs as below (example in /tmp/hive/hive.log).
> 2016-08-22 06:43:47,655 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1177 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,756 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1178 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,858 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1179 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:47,961 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1180 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,062 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1181 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,165 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1182 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,265 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1183 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,366 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1184 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
> 2016-08-22 06:43:48,467 WARN [kafka-producer-network-thread | producer-1]:
> clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while
> fetching metadata with correlation id 1185 :
> {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
