[kafka] branch 3.1 updated: KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled (#11939)

[ijuma]

This is an automated email from the ASF dual-hosted git repository.

ijuma pushed a commit to branch 3.1
in repository https://gitbox.apache.org/repos/asf/kafka.git


The following commit(s) were added to refs/heads/3.1 by this push:
     new 8e1edef  KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is 
enabled (#11939)
8e1edef is described below

commit 8e1edef9bfdfb6013c473fb61dfeb23c479dfd8d
Author: Yang Yu <y...@confluent.io>
AuthorDate: Thu Mar 24 08:36:56 2022 -0500

    KAFKA-13761: KafkaLog4jAppender deadlocks when idempotence is enabled 
(#11939)
    
    When a log entry is appended to a Kafka topic using KafkaLog4jAppender, the 
producer.send operation
    may hit a deadlock if the producer network thread also tries to append a 
log at the same log level.
    This issue is triggered when idempotence is enabled for the 
KafkaLog4jAppender and the producer
    tries to acquire the TransactionManager lock.
    
    This is a temporary workaround to avoid deadlocks by disabling idempotence 
explicitly in
    KafkaLog4jAppender.
    
    Reviewers: Luke Chen <show...@gmail.com>, Ismael Juma <ism...@juma.me.uk>
---
 .../main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java  | 4 ++++
 1 file changed, 4 insertions(+)

diff --git 
a/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java
 
b/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java
index 23272a2..c561fc2 100644
--- 
a/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java
+++ 
b/log4j-appender/src/main/java/org/apache/kafka/log4jappender/KafkaLog4jAppender.java
@@ -43,6 +43,7 @@ import static 
org.apache.kafka.clients.producer.ProducerConfig.LINGER_MS_CONFIG;
 import static 
org.apache.kafka.clients.producer.ProducerConfig.MAX_BLOCK_MS_CONFIG;
 import static org.apache.kafka.clients.producer.ProducerConfig.RETRIES_CONFIG;
 import static 
org.apache.kafka.clients.producer.ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG;
+import static 
org.apache.kafka.clients.producer.ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG;
 import static org.apache.kafka.common.config.SaslConfigs.SASL_JAAS_CONFIG;
 import static 
org.apache.kafka.common.config.SaslConfigs.SASL_KERBEROS_SERVICE_NAME;
 import static org.apache.kafka.common.config.SaslConfigs.SASL_MECHANISM;
@@ -290,6 +291,9 @@ public class KafkaLog4jAppender extends AppenderSkeleton {
         props.put(DELIVERY_TIMEOUT_MS_CONFIG, deliveryTimeoutMs);
         props.put(LINGER_MS_CONFIG, lingerMs);
         props.put(BATCH_SIZE_CONFIG, batchSize);
+        // Disable idempotence to avoid deadlock when the producer network 
thread writes a log line while interacting
+        // with the TransactionManager, see KAFKA-13761 for more information.
+        props.put(ENABLE_IDEMPOTENCE_CONFIG, false);
 
         if (securityProtocol != null) {
             props.put(SECURITY_PROTOCOL_CONFIG, securityProtocol);