Hi Andreas,
I just tried running a Kafka 3.6 client on a recent Temurin Jdk 8
(OpenJDK8U-jdk_x64_mac_hotspot_8u392b08) and configured it with
clientProps.put(SslConfigs.SSL_PROTOCOL_CONFIG,"TLSv1.3");
clientProps.put(SslConfigs.SSL_ENABLED_PROTOCOLS_CONFIG,"TLSv1.3");
and it connects without issues for me
On Mon, 6 Nov 2023 at 11:50, Andreas Martens1 <amart...@uk.ibm.com> wrote:
>
> Hi Edoardo,
>
> Yes, that’s exactly what I’m saying!
>
> If I run the console consumer in the build with:
> kafka-console-consumer.sh --bootstrap-server <server>:443 --topic <topic>
> --consumer.config $PWD/consumer.properties
>
> Where consumer.properties contains:
> sasl.mechanism=PLAIN
> security.protocol=SASL_SSL
> sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule
> required \
> username="<username>" \
> password="<password>";
>
> ssl.truststore.location=truststore.jks
> ssl.truststore.password=changeit
> ssl.truststore.type=JKS
>
> ssl.enabled.protocols=TLSv1.3
>
> With Java 11 it works fine, but with Java 1.8 I get:
> [main] ERROR kafka.tools.ConsoleConsumer$ - Error processing message,
> terminating consumer process:
> org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake
> failed
> Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol
> (protocol is disabled or cipher suites are inappropriate)
> at com.ibm.jsse2.aa.<init>(aa.java:9)
> at com.ibm.jsse2.ab.<init>(ab.java:44)
> at com.ibm.jsse2.bb.a(bb.java:129)
> at com.ibm.jsse2.bg.beginHandshake(bg.java:141)
> at
> org.apache.kafka.common.network.SslTransportLayer.startHandshake(SslTransportLayer.java:125)
> at
> org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:274)
> at
> org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178)
> at
> org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543)
> at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
> at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:571)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:281)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:231)
> at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:274)
> at
> org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:248)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnreadySync(ConsumerCoordinator.java:443)
> at
> org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:475)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1296)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1255)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
> at
> kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:473)
> at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:103)
> at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:77)
> at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:54)
> at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)
>
> If I change the test in SslConfigs.java to test for presence of the TPSv1.3
> protocol rather than testing for Java version, then both versions connect OK.
>
> Thanks,
> Andreas
>
>
>
> From: Edoardo Comar <edoardli...@gmail.com>
> Date: Friday, 3 November 2023 at 17:26
> To: users@kafka.apache.org <users@kafka.apache.org>
> Cc: d...@kafka.apache.org <d...@kafka.apache.org>
> Subject: [EXTERNAL] Re: Java 1.8 and TLSv1.3
> Andreas,
> do you mean that even if you configure your Java client running on Java8 with
> ssl.enabled.protocols=TLSv1.3
> you can't connect to a Kafka broker using TLS1.3 ?
>
>
> On Sat, 28 Oct 2023 at 01:03, Ismael Juma <m...@ismaeljuma.com> wrote:
> >
> > Hi Andreas,
> >
> > The TLS code has run into changes in behavior across different Java
> > versions, so we only wanted to allow TLS 1.3 in the versions we tested
> > against. TLS 1.3 landed in Java 8 a while after we made the relevant
> > changes for Java 11 and newer. That said, Java 8 support is deprecated and
> > will be removed in Apache Kafka 4.0 (in a few months). So, it's not clear
> > we want to invest in making more features available for that version.
> >
> > Thanks,
> > Ismael
> >
> > On Thu, Oct 26, 2023 at 4:49 AM Andreas Martens1 <amart...@uk.ibm.com>
> > wrote:
> >
> > > Hello good people of Kafka,
> > >
> > > I was recently informed that TLS 1.3 doesn’t work for connecting our
> > > product to Kafka, and after some digging realised it was true, no matter
> > > how hard I type “TLSv1.3” it doesn’t work, weirdly with an error about no
> > > applicable Ciphers.
> > >
> > > So after a bunch more digging I realised that the problem lies in the
> > > Kafka client classes, in Kafka clients’ SslConfigs.java there is this
> > > code:
> > > static {
> > > if (Java.IS_JAVA11_COMPATIBLE) {
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.3";
> > > DEFAULT_SSL_ENABLED_PROTOCOLS = "TLSv1.2,TLSv1.3";
> > > } else {
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.2";
> > > DEFAULT_SSL_ENABLED_PROTOCOLS = "TLSv1.2";
> > > }
> > > }
> > >
> > > My initial thoughts were that these just set the defaults, I should still
> > > be able to set TLSv1.3 in my properties, but no. If I change the above
> > > block to:
> > > static {
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.3";
> > > DEFAULT_SSL_ENABLED_PROTOCOLS = "TLSv1.2,TLSv1.3";
> > > }
> > > it works just fine. I suspect (but haven’t yet had the time to verify)
> > > that there’s something that gets the list of supported Ciphers from the
> > > default, and applies those Ciphers to the selected end protocol, and since
> > > there’s no overlap I’m outta luck.
> > >
> > > Now of course life is never simple, so I can’t just make the above change
> > > and send you fine people a PR, since there’s probably still some older
> > > JREs
> > > out there that don’t support TLSv1.3.
> > >
> > > As a fine person once told me (actually, a Java support person) “don’t
> > > test the symptom, test the cause”. In this case, we shouldn’t be testing
> > > whether we’re working with a Java 11 JVM, we should test whether our
> > > current JVM has a TLSv1.3 Context instance. E.g.:
> > > try{
> > > SSLContext context = SSLContext.getInstance("TLSv1.3");
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.3";
> > > }
> > > catch(java.security.NoSuchAlgorithmException e){
> > > DEFAULT_SSL_PROTOCOL = "TLSv1.2";
> > > }
> > > But the test in SslConfigs.java is done in *static* init, and as we all
> > > know, performing try-catch within a static is a Big No-No.
> > >
> > > Soooo, before I go digging further in the code and start looking to modify
> > > the places where the defaults are consumed, does anyone have a better
> > > idea?
> > > My initial thought was to raise a ticket and run away, but I’m trying to
> > > be
> > > a good citizen!
> > >
> > > I should probably mention, this code was introduced in
> > > https://issues.apache.org/jira/browse/KAFKA-7251 and
> > > https://issues.apache.org/jira/browse/KAFKA-9320 and
> > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-573%3A+Enable+TLSv1.3+by+default
> > > .
> > >
> > > Thanks,
> > > Andreas Martens
> > > --
> > > Senior Engineer
> > > IBM App Connect Enterprise
> > >
> > > Unless otherwise stated above:
> > >
> > > IBM United Kingdom Limited
> > > Registered in England and Wales with number 741598
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
> > >
>
> Unless otherwise stated above:
>
> IBM United Kingdom Limited
> Registered in England and Wales with number 741598
> Registered office: PO Box 41, North Harbour, Portsmouth, Hants. PO6 3AU
