Hi seems you setup in port 9093 only ssl as a method of authentication and method of transfer encryption, so it means in the client configuration you would need the keystore configured as well, you could choose other mean of authentication such as PLAIN_SSL or so own, hope thats helps, keep us updated, good luck
בתאריך יום ו׳, 30 באפר׳ 2021, 19:27, מאת Calvin Chen <pingc...@hotmail.com >: > Hi all > > I'm working on Kafka(kafka_2.13-2.7.0)cluster with SSL enabled, and I need > help on Kafka broker config(I got error of connection failed) and client > SSL config(I got error of SSL handshake failed). > > > I setup Kafka and client SSL config by taking reference of > Apache Kafka<https://kafka.apache.org/documentation/#security_ssl> > Apache Kafka TLS encryption & authentication - Azure HDInsight | Microsoft > Docs< > https://docs.microsoft.com/en-us/azure/hdinsight/kafka/apache-kafka-ssl-encryption-authentication > > > > And I can verify my Kafka cluster SSL with below command: > > openssl s_client -debug -connect sc2-kafka-dev-001_node-1:9093 -tls1_2 > > some output is: > > Server certificate > -----BEGIN CERTIFICATE----- > MIID1TCCAb0CFGy5db0MHYKTnZZAQpnHsR3ywrsqMA0GCSqGSIb3DQEBCwUAMBwx > GjAYBgNVBAMMEUthZmthLVNlY3VyaXR5LUNBMB4XDTIxMDQzMDE0NDEzMVoXDTIy > MDQzMDE0NDEzMVowMjEwMC4GA1UEAwwnc2MyLWthZmthLWRldi0wMDFfbm9kZS0x > LmVuZy52bXdhcmUuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA > wuL14qBmI++Ii/lxLU32TlGd0VlDX29JXjyqEUoaXDjYBroY5+FDhawladB3YU3/ > IY2fQ9PHoPLVntBnMMf29m8buVFKXsRT0mOjkyVuUUZcp0L9mLMKnKE1Rn+EJM93 > Ys0A8/YJgp3LYu0cbLbqw9TUdFkyesaV5zqAXse14npi0eqXk5pk5ss2ePfqa6bN > m2zM1eZrJjjp1vFx0oL8N6z2z6+AS67unyj9x2SjyXQgigbnz36VM99EUeMeQLuz > weuZN97sKKW4ub+ya0R6lbS5pum+iQ4ukA9TeiXllqwoFZTEZistsbec5OvgVgC0 > 41I6rtlGdqkAPEyU8xtfnwIDAQABMA0GCSqGSIb3DQEBCwUAA4ICAQAdTBndO51t > IK40oYHf2dWHE4WPvZfDoQpAVwhLptsbQD4RVdpPUxagbh4F4zAFwIZgCpwU0YBz > sq71p45x/3NjX40eIWsC0WgQoCQsCWimXQSMOltopNEhrSICd7mD1H/C1uftNXU1 > uAGRGUC8wgX1ULdHLg0Szvz519ia+uZqOKyzsMBDZnmtesli3lTmXjjO5E5aPLaU > ztLeZrhHzR7ib9ZtIidl4hviPKbdLBPkeBqk7b821RbCK1Ny8eSOBYY3wePqTGU3 > LbLEEeFgNBr9wEsmEcr237QW4UrYX5TjxeoykQj72u9tAb8mTrAY8QXUo9f826hQ > kTcSe504t6hMmX6oP9R3wUHqpIAZ3woqOV/I2KwCt2L3thUXyJK7F9XTSZQq89DT > E4SQlEthR+Mq/eIqyunq403MnQuxRGpfkiOLzBO1vUYDbnWjaC3oouTW9Y1rhF0L > t+DqaMXSTLyhcLZ8xUMcpgfROMArjufTsQ5KWqUYCTUffsrRVFzlyg02OjzgYJ5a > XR/lp64V3Ul1/8EM7QujDgdq9KTRu4FxuOk+8AFMOz4UJ1iqFONBKz6UTYmKjECw > aEp8k8WjuyHeuO5+d9qav+xYSQbHhZ5QSILKlyDSDkLWTjgNyvCMKzabtTW1HfQJ > p4DsCTjGse76yHJNAnH0jdGBVvi8ONdhuA== > -----END CERTIFICATE----- > subject=CN = sc2-kafka-dev-001_node-1.eng.vmware.com > > issuer=CN = Kafka-Security-CA > > > So when I see above output, does it means my SSL setup for Kafka broker is > ok? > > > However, I didn't get below keyword in server.log, as mentioned from Kafka > webpage, I should see below in server.log. > > > with addresses: PLAINTEXT -> EndPoint({{fqdn}},9092,PLAINTEXT),SSL -> > EndPoint({{fqdn}},9093,SSL) > > My two server.log output are: > > [2021-04-30 09:05:08,954] INFO [KafkaServer id=1] started > (kafka.server.KafkaServer) > > While another one is: > > [2021-04-30 09:05:30,183] WARN [Controller id=2, targetBrokerId=1] > Connection to node 1 ( > sc2-kafka-dev-001_node-1.eng.vmware.com/10.185.50.10:9093) could not be > established. Broker may not be available. > (org.apache.kafka.clients.NetworkClient) > [2021-04-30 09:05:30,311] WARN [Controller id=2, targetBrokerId=3] > Connection to node 3 ( > sc2-kafka-dev-001_node-3.eng.vmware.com/10.185.50.12:9093) could not be > established. Broker may not be available. > (org.apache.kafka.clients.NetworkClient) > > It looks like the Kafka cluster with SSL enabled has some problem on setup > connection across brokers. BTW, I haven't apply for the DNS record for my > brokers, I setup domain name in /etc/hosts, and it shall be ok for the test? > > > Also, when I test Kafka command line with SSL config, I see auth error, > but I didn't config auth, I just config ssl encryption: > > [worker@sc2-kafka-dev-001_node-1 client]$ > /opt/kafka/kafka_2.13-2.7.0/bin/kafka-console-producer.sh --broker-list > sc2-kafka-dev-001_node-1:9093 --topic topic1 --producer.config > ./client-ssl.properties > >[2021-04-30 09:11:19,574] ERROR [Producer clientId=console-producer] > Connection to node -1 (sc2-kafka-dev-001_node-1/10.185.50.10:9093) failed > authentication due to: SSL handshake failed > (org.apache.kafka.clients.NetworkClient) > [2021-04-30 09:11:19,575] WARN [Producer clientId=console-producer] > Bootstrap broker sc2-kafka-dev-001_node-1:9093 (id: -1 rack: null) > disconnected (org.apache.kafka.clients.NetworkClient) > > > Here is my part of Kafka broker config: > > listeners=PLAINTEXT://sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL:// > sc2-kafka-dev-001_node-2.eng.vmware.com:9093 > advertised.listeners=PLAINTEXT:// > sc2-kafka-dev-001_node-2.eng.vmware.com:9092, SSL:// > sc2-kafka-dev-001_node-2.eng.vmware.com:9093 > > ssl.endpoint.identification.algorithm= > security.inter.broker.protocol=SSL > > ssl.keystore.location=/data/ssl1/kafka.server.keystore.jks > ssl.keystore.password=MyServerPassword123 > ssl.key.password=MyServerPassword123 > ssl.truststore.location=/data/ssl1/kafka.server.truststore.jks > ssl.truststore.password=MyServerPassword123 > ssl.enabled.protocols=TLSv1.2 > ssl.truststore.type=JKS > ssl.keystore.type=JKS > ssl.secure.random.implementation=SHA1PRNG > > > Here is my client config: > > security.protocol=SSL > ssl.truststore.location=/data/client/kafka.client.truststore.jks > ssl.truststore.password=MyClientPassword123 > ssl.enabled.protocols=TLSv1.2 > > > > THANKS >
