Ahh, that’s it, the second property needs to be set too, we weren’t doing that!
By setting both ssl.protocol and ssl.enabled.protocols we can get TLSv1.3.
Annoyingly the JRE implementations differ, some interpret “TLSv1.3” to mean
“TLS 1.3 and TLS 1.2” and some to mean “TLS 1.3 only”. I can’t see any way to
specify, across JRE implementations, “allow both TLS 1.2 and 1.3”.
Thanks
Andreas
From: Edoardo Comar <edoardli...@gmail.com>
Date: Monday, 6 November 2023 at 12:43
To: us...@kafka.apache.org <us...@kafka.apache.org>
Cc: dev@kafka.apache.org <dev@kafka.apache.org>
Subject: [EXTERNAL] Re: Java 1.8 and TLSv1.3
Hi Andreas,
I just tried running a Kafka 3.6 client on a recent Temurin Jdk 8
(OpenJDK8U-jdk_x64_mac_hotspot_8u392b08) and configured it with
clientProps.put(SslConfigs.SSL_PROTOCOL_CONFIG,"TLSv1.3");
clientProps.put(SslConfigs.SSL_ENABLED_PROTOCOLS_CONFIG,"TLSv1.3");
and it connects without issues for me
On Mon, 6 Nov 2023 at 11:50, Andreas Martens1 <amart...@uk.ibm.com> wrote:
>
> Hi Edoardo,
>
> Yes, that’s exactly what I’m saying!
>
> If I run the console consumer in the build with:
> kafka-console-consumer.sh --bootstrap-server <server>:443 --topic <topic>
> --consumer.config $PWD/consumer.properties
>
> Where consumer.properties contains:
> sasl.mechanism=PLAIN
> security.protocol=SASL_SSL
> sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
> required \
> username="<username>" \
> password="<password>";
>
> ssl.truststore.location=truststore.jks
> ssl.truststore.password=changeit
> ssl.truststore.type=JKS
>
> ssl.enabled.protocols=TLSv1.3
>
> With Java 11 it works fine, but with Java 1.8 I get:
> [main] ERROR kafka.tools.ConsoleConsumer$ - Error processing message,
> terminating consumer process:
> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> failed
> Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol
> (protocol is disabled or cipher suites are inappropriate)
> at com.ibm.jsse2.aa.<init>(aa.java:9)
> at com.ibm.jsse2.ab.<init>(ab.java:44)
> at com.ibm.jsse2.bb.a(bb.java:129)
> at com.ibm.jsse2.bg.beginHandshake(bg.java:141)
> at
> org.apache.kafka.common.network.SslTransportLayer.startHandshake(SslTransportLayer.java:125)
> at
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:274)
> at
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178)
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:571)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:281)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:231)
> at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:274)
> at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:248)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnreadySync(ConsumerCoordinator.java:443)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:475)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1296)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1255)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
> at
> kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:473)
> at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:103)
> at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:77)
> at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:54)
> at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
>
> If I change the test in SslConfigs.java to test for presence of the TPSv1.3
> protocol rather than testing for Java version, then both versions connect OK.
>
> Thanks,
> Andreas
>
>
>
> From: Edoardo Comar <edoardli...@gmail.com>
> Date: Friday, 3 November 2023 at 17:26
> To: us...@kafka.apache.org <us...@kafka.apache.org>
> Cc: dev@kafka.apache.org <dev@kafka.apache.org>
> Subject: [EXTERNAL] Re: Java 1.8 and TLSv1.3
> Andreas,
> do you mean that even if you configure your Java client running on Java8 with
> ssl.enabled.protocols=TLSv1.3
> you can't connect to a Kafka broker using TLS1.3 ?
>
>
> On Sat, 28 Oct 2023 at 01:03, Ismael Juma <m...@ismaeljuma.com> wrote:
> >
> > Hi Andreas,
> >
> > The TLS code has run into changes in behavior across different Java
> > versions, so we only wanted to allow TLS 1.3 in the versions we tested
> > against. TLS 1.3 landed in Java 8 a while after we made the relevant
> > changes for Java 11 and newer. That said, Java 8 support is deprecated and
> > will be removed in Apache Kafka 4.0 (in a few months). So, it's not clear
> > we want to invest in making more features available for that version.
> >
> > Thanks,
> > Ismael
> >
> > On Thu, Oct 26, 2023 at 4:49 AM Andreas Martens1 <amart...@uk.ibm.com>
> > wrote:
> >
> > > Hello good people of Kafka,
> > >
> > > I was recently informed that TLS 1.3 doesn’t work for connecting our
> > > product to Kafka, and after some digging realised it was true, no matter
> > > how hard I type “TLSv1.3” it doesn’t work, weirdly with an error about no
> > > applicable Ciphers.
> > >
> > > So after a bunch more digging I realised that the problem lies in the
> > > Kafka client classes, in Kafka clients’ SslConfigs.java there is this
> > > code:
> > > static {
> > > if (Java.IS_JAVA11_COMPATIBLE) {
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.3";
> > > DEFAULT_SSL_ENABLED_PROTOCOLS = "TLSv1.2,TLSv1.3";
> > > } else {
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.2";
> > > DEFAULT_SSL_ENABLED_PROTOCOLS = "TLSv1.2";
> > > }
> > > }
> > >
> > > My initial thoughts were that these just set the defaults, I should still
> > > be able to set TLSv1.3 in my properties, but no. If I change the above
> > > block to:
> > > static {
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.3";
> > > DEFAULT_SSL_ENABLED_PROTOCOLS = "TLSv1.2,TLSv1.3";
> > > }
> > > it works just fine. I suspect (but haven’t yet had the time to verify)
> > > that there’s something that gets the list of supported Ciphers from the
> > > default, and applies those Ciphers to the selected end protocol, and since
> > > there’s no overlap I’m outta luck.
> > >
> > > Now of course life is never simple, so I can’t just make the above change
> > > and send you fine people a PR, since there’s probably still some older
> > > JREs
> > > out there that don’t support TLSv1.3.
> > >
> > > As a fine person once told me (actually, a Java support person) “don’t
> > > test the symptom, test the cause”. In this case, we shouldn’t be testing
> > > whether we’re working with a Java 11 JVM, we should test whether our
> > > current JVM has a TLSv1.3 Context instance. E.g.:
> > > try{
> > > SSLContext context = SSLContext.getInstance("TLSv1.3");
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.3";
> > > }
> > > catch(java.security.NoSuchAlgorithmException e){
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.2";
> > > }
> > > But the test in SslConfigs.java is done in *static* init, and as we all
> > > know, performing try-catch within a static is a Big No-No.
> > >
> > > Soooo, before I go digging further in the code and start looking to modify
> > > the places where the defaults are consumed, does anyone have a better
> > > idea?
> > > My initial thought was to raise a ticket and run away, but I’m trying to
> > > be
> > > a good citizen!
> > >
> > > I should probably mention, this code was introduced in
> > > https://issues.apache.org/jira/browse/KAFKA-7251 and
> > > https://issues.apache.org/jira/browse/KAFKA-9320 and
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-573%3A+Enable+TLSv1.3+by+default
> > > .
> > >
> > > Thanks,
> > > Andreas Martens
> > > --
> > > Senior Engineer
> > > IBM App Connect Enterprise
> > >
> > > Unless otherwise stated above:
> > >
> > > IBM United Kingdom Limited
> > > Registered in England and Wales with number 741598
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
> > >
>
> Unless otherwise stated above:
>
> IBM United Kingdom Limited
> Registered in England and Wales with number 741598
> Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
Unless otherwise stated above:
IBM United Kingdom Limited
Registered in England and Wales with number 741598
Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
