Jakub Scholz created KAFKA-20437:
------------------------------------

             Summary: Dynamic update of ssl.enabled.protocols does not seem to 
take effect
                 Key: KAFKA-20437
                 URL: https://issues.apache.org/jira/browse/KAFKA-20437
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 4.1.1, 4.2.0
            Reporter: Jakub Scholz


According to the 
[documentation|https://kafka.apache.org/42/configuration/broker-configs/#brokerconfigs_ssl.enabled.protocols],
 the `ssl.enabled.protocols` configuration option is dynamically configurable 
on a per-broker basis. However, it seems that it is not the case. When you try 
to update it, Kafka seemingly accepts the configuration change and reports 
success. But it ignores the change and the change is not applied unless you 
change it in the configuration file and restart the brokers.

This is pretty dangerous, given that this is a security option and might lead 
to users thinking their security configuration, for example, allows only 
TLSv1.3 connections, while in reality it also allows TLSv1.2. And given the 
number of security regulations that various Kafka users might need to follow, 
this can easily be a regulatory challenge as well.

So I think this should be fixed in the configuration model and in the 
documentation. Trying to dynamically configure this option should also be 
rejected by Kafka.

_(I was able to reproduce this with Kafka 4.2.0 and 4.1.1. But I assume this 
affects older Kafka versions as well.)_
----
Steps to reproduce:

1. Deploy a simple Kafka cluster with a single mixed role node following the 
quickstart ([https://kafka.apache.org/quickstart/])

2. Before starting the Kafka node, edit the server.properties and add a new 
SSL-based listener and configure a server certificate. The following lines are 
what I changed in {{{}server.properties{}}}:
{code:java}
listeners=PLAINTEXT://:9092,CONTROLLER://:9093,SSL://localhost:9094
advertised.listeners=PLAINTEXT://localhost:9092,CONTROLLER://localhost:9093,SSL://localhost:9094
ssl.client.auth=none
ssl.keystore.location=/my/path/server-1.keystore
ssl.keystore.password=123456{code}
3. Start the Kafka node with bin/kafka-server-start.sh config/server.properties

4. Try to connect with OpenSSL (You can also use Kafka client if you want) 
using TLSv1.2 => works as expected
{code:java}
$ openssl s_client -connect localhost:9094 -tls1_2
Connecting to 127.0.0.1
CONNECTED(00000005)
...
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Protocol: TLSv1.2
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
...{code}
5. Try to connect with OpenSSL (You can also use Kafka client if you want) 
using TLSv1.3 => works as expected
{code:java}
$ openssl s_client -connect localhost:9094 -tls1_3
Connecting to 127.0.0.1
CONNECTED(00000005)
...
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Protocol: TLSv1.3
Server public key is 2048 bit
This TLS version forbids renegotiation.
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 19 (self-signed certificate in certificate chain)
---
---
Post-Handshake New Session Ticket arrived:
SSL-Session:
    Protocol  : TLSv1.3
    Cipher    : TLS_AES_256_GCM_SHA384
...{code}
6. Change the ssl.enabled.protocols to TLSv1.3
{code:java}
$ bin/kafka-configs.sh --bootstrap-server localhost:9092 \
  --entity-type brokers \                                                       
                                                                                
                                                      --entity-name 1 \
  --alter \
  --add-config "ssl.enabled.protocols=TLSv1.3"
Completed updating config for broker 1.{code}
7. The Kafka log seems to reflect the change:
{code:java}
[2026-03-31 23:00:50,024] INFO [DynamicConfigPublisher controller id=1] 
Updating node 1 with new configuration : ssl.enabled.protocols -> TLSv1.3 
(kafka.server.metadata.DynamicConfigPublisher)
[2026-03-31 23:00:50,026] INFO KafkaConfig values: 
        ...
        ssl.allow.dn.changes = false
        ssl.allow.san.changes = false
        ssl.cipher.suites = []
        ssl.client.auth = none
        ssl.enabled.protocols = [TLSv1.3]
        ssl.endpoint.identification.algorithm = https
        ssl.engine.factory.class = null
        ssl.key.password = null
        ssl.keymanager.algorithm = SunX509
        ssl.keystore.certificate.chain = null
        ssl.keystore.key = null
        ssl.keystore.location = 
/Users/scholzj/development/my-kafka-client-sandbox/ssl-ca/keys/server-0.keystore
        ssl.keystore.password = [hidden]
        ssl.keystore.type = JKS
        ssl.principal.mapping.rules = DEFAULT
        ssl.protocol = TLSv1.3
        ssl.provider = null
        ssl.secure.random.implementation = null
        ssl.trustmanager.algorithm = PKIX
        ssl.truststore.certificates = null
        ssl.truststore.location = null
        ssl.truststore.password = null
        ssl.truststore.type = JKS
        ...
 (org.apache.kafka.common.config.AbstractConfig)
[2026-03-31 23:00:50,034] INFO [DynamicConfigPublisher broker id=1] Updating 
node 1 with new configuration : ssl.enabled.protocols -> TLSv1.3 
(kafka.server.metadata.DynamicConfigPublisher)
[2026-03-31 23:00:50,035] INFO KafkaConfig values: 
        ...
        ssl.allow.dn.changes = false
        ssl.allow.san.changes = false
        ssl.cipher.suites = []
        ssl.client.auth = none
        ssl.enabled.protocols = [TLSv1.3]
        ssl.endpoint.identification.algorithm = https
        ssl.engine.factory.class = null
        ssl.key.password = null
        ssl.keymanager.algorithm = SunX509
        ssl.keystore.certificate.chain = null
        ssl.keystore.key = null
        ssl.keystore.location = 
/Users/scholzj/development/my-kafka-client-sandbox/ssl-ca/keys/server-0.keystore
        ssl.keystore.password = [hidden]
        ssl.keystore.type = JKS
        ssl.principal.mapping.rules = DEFAULT
        ssl.protocol = TLSv1.3
        ssl.provider = null
        ssl.secure.random.implementation = null
        ssl.trustmanager.algorithm = PKIX
        ssl.truststore.certificates = null
        ssl.truststore.location = null
        ssl.truststore.password = null
        ssl.truststore.type = JKS
        ...
 (org.apache.kafka.common.config.AbstractConfig)
[2026-03-31 23:00:50,041] INFO Reconfiguring 
org.apache.kafka.common.network.SslChannelBuilder@2c7bed74, updated configs: 
Set() custom configs: {} (kafka.server.DynamicBrokerConfig)
[2026-03-31 23:00:50,043] INFO Reconfiguring 
org.apache.kafka.common.network.SslChannelBuilder@4ac39bd6, updated configs: 
Set() custom configs: {} (kafka.server.DynamicBrokerConfig)
[2026-03-31 23:00:50,045] INFO Reconfiguring 
org.apache.kafka.common.network.SslChannelBuilder@211381c9, updated configs: 
Set() custom configs: {} (kafka.server.DynamicBrokerConfig){code}
8. Try to connect with OpenSSL (You can also use Kafka client if you want) 
using TLSv1.2 => Still works, *NOT EXPECTED*
{code:java}
openssl s_client -connect localhost:9094 -tls1_2        
Connecting to 127.0.0.1
...
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Protocol: TLSv1.2
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
...{code}
9. Only after I add ssl.enabled.protocols=TLSv1.3 to the server.properties file 
and restart the Kafka node, TLSv1.2 stops working as expected:
{code:java}
$ openssl s_client -connect localhost:9094 -tls1_2
Connecting to 127.0.0.1
CONNECTED(00000005)
00716C0B02000000:error:0A00042E:SSL routines:ssl3_read_bytes:tlsv1 alert 
protocol version:ssl/record/rec_layer_s3.c:918:SSL alert number 70
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 189 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Protocol: TLSv1.2
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : 0000
    Session-ID: 
    Session-ID-ctx: 
    Master-Key: 
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1774991157
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
00716C0B02000000:error:0A000197:SSL routines:SSL_shutdown:shutdown while in 
init:ssl/ssl_lib.c:2804:{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to