[
https://issues.apache.org/jira/browse/KAFKA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16523836#comment-16523836
]
Bastien Bussy commented on KAFKA-7099:
--------------------------------------
Hello,
I am a colleague of Vincent.
*Disclaimer* : we work on hadoop (HDP-2.6.4) with *log4j 1.2.17*
After some digging, the problem is caused by the
"kafka-producer-network-thread" which is in charge of updating the metadata.
In this process, the Sender executes a poll on the NetworkClient, which
executes the maybeUpdate method.
In this method, there is a call to log.debug but there is a synchronized
mecanism in log4j.Category.callAppenders.
So, *there is a thread deadlock* between the main thread where we want to use
the Kafka Appender and the network-thread.
This is the stacktrace from the network-thread :
{code:java}
at org.apache.log4j.Category.callAppenders:204
at org.apache.log4j.Category.forcedLog:391
at org.apache.log4j.Category.log:856
at org.slf4j.impl.Log4jLoggerAdapter.debug:230
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate:644
at org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater.maybeUpdate:552
at org.apache.kafka.clients.NetworkClient.poll:258
at org.apache.kafka.clients.producer.internals.Sender.run:236
at org.apache.kafka.clients.producer.internals.Sender.run:135
at java.lang.Thread.run:748
{code}
To fix this issue temporarly, we force the level on logger _org.apache.kafka_
to INFO when we have the root logger in DEBUG mode.
I think this is a common issue in Log4j 1.2.
Do you have any recommendations?
> KafkaLog4jAppender - not sending any message with level DEBUG
> -------------------------------------------------------------
>
> Key: KAFKA-7099
> URL: https://issues.apache.org/jira/browse/KAFKA-7099
> Project: Kafka
> Issue Type: Bug
> Components: log
> Affects Versions: 0.10.2.0
> Reporter: Vincent Lebreil
> Priority: Major
>
> KafkaLog4jAppender can be stuck if it is defined at root category with level
> DEBUG
> {{log4j.rootLogger=DEBUG, kafka}}
> {{log4j.appender.kafka=org.apache.kafka.log4jappender.KafkaLog4jAppender}}
> {quote}DEBUG org.apache.kafka.clients.producer.KafkaProducer:131 - Exception
> occurred during message send:
> org.apache.kafka.common.errors.TimeoutException: Failed to update metadata
> after 60000 ms.
> {quote}
> KafkaLog4jAppender is using a KafkaProducer using itself Log4j with messages
> at levels TRACE and DEBUG. The appender used in this case is also the
> KafkaLog4jAppender.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
