server.properties ######
broker.id=11 port=9093 host.name=n1 advertised.host.name=192.168.0.11 allow.everyone.if.no.acl.found=true super.users=User:CN=n1.test.com,OU=TEST,O=TEST,L=TEST,ST=TEST,C=TEST listeners=SSL://n1.test.com:9093 <http://n1.test.com:9092/> advertised.listeners=SSL://n1.test.com:9093 <http://n1.test.com:9092/> ssl.client.auth=required ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1 ssl.keystore.type=JKS ssl.truststore.type=JKS security.inter.broker.protocol=SSL ssl.keystore.location=/home/kafka/kafka.server.keystore.jks ssl.keystore.password=Test2017 ssl.key.password=Test2017 ssl.truststore.location=/home/kafka/kafka.server.truststore.jks ssl.truststore.password=Test2017 authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer principal.builder.class=org.apache.kafka.common.security.auth.DefaultPrincipalBuilder num.replica.fetchers=4 replica.fetch.max.bytes=1048576 replica.fetch.wait.max.ms=500 replica.high.watermark.checkpoint.interval.ms=5000 replica.socket.timeout.ms=30000 replica.socket.receive.buffer.bytes=65536 replica.lag.time.max.ms=10000 controller.socket.timeout.ms=30000 controller.message.queue.size=10 default.replication.factor=3 log.dirs=/usr/log/kafka kafka.logs.dir=/usr/log/kafka num.partitions=20 message.max.bytes=1000000 auto.create.topics.enable=true log.index.interval.bytes=4096 log.index.size.max.bytes=10485760 log.retention.hours=720 log.flush.interval.ms=10000 log.flush.interval.messages=20000 log.flush.scheduler.interval.ms=2000 log.roll.hours=168 log.retention.check.interval.ms=300000 log.segment.bytes=1073741824 delete.topic.enable=true socket.request.max.bytes=104857600 socket.receive.buffer.bytes=1048576 socket.send.buffer.bytes=1048576 num.io.threads=8 num.network.threads=8 queued.max.requests=16 fetch.purgatory.purge.interval.requests=100 producer.purgatory.purge.interval.requests=100 zookeeper.connect=n1:2181,n2:2181,n3:2181 zookeeper.connection.timeout.ms=2000 zookeeper.sync.time.ms=2000 ###### producer.properties ###### bootstrap.servers=n1.test.com:9093 <http://n1.test.com:9092/> security.protocol=SSL ssl.truststore.location=/home/kafka/kafka.client.truststore.jks ssl.truststore.password=testkafka ssl.keystore.location=/home/kafka/kafka.client.keystore.jks ssl.keystore.password=testkafka ssl.key.password=testkafka ##### (I had tried to switch to another port, 9093 is the correct port) On Thu, Aug 10, 2017 at 4:28 AM, M. Manna <manme...@gmail.com> wrote: > Your openssl test is showing connected with port 9092. but your previous > messages show 9093 - is there some typo issues? Where is SSL running > > Please share the following and don't leave any details out. This will only > create more assumptions. > > 1) server.properties > 2) Zookeeper.properties > > Also, run the following command (when the cluster is running) > zookeeper-shell.sh localhost:2181 > get /brokers/ids/11 > > Does it show that your broker #11 is connected? > > > > > On 9 August 2017 at 21:17, Ascot Moss <ascot.m...@gmail.com> wrote: > > > Dear Manna, > > > > > > What's the status of your SSL? Have you verified that the setup is > working? > > Yes, I used " > > > > openssl s_client -debug -connect n1.test.com:9092 -tls1 > > Output: > > > > CONNECTED(00000003) > > > > write to 0x853e70 [0x89fd43] (155 bytes => 155 (0x9B)) > > > > 0000 - 16 03 01 00 96 01 00 00-92 03 01 59 8b 6d 0d b1 ...........Y.m.. > > ... > > > > Server certificate > > > > -----BEGIN CERTIFICATE----- > > > > CwwCSEsxGT............ > > > > -----END CERTIFICATE----- > > > > --- > > > > SSL handshake has read 2470 bytes and written 161 bytes > > > > --- > > > > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA > > > > PSK identity hint: None > > > > Start Time: 1502309645 > > > > Timeout : 7200 (sec) > > > > Verify return code: 19 (self signed certificate in certificate chain) > > > > --- > > > > Regards > > > > On Wed, Aug 9, 2017 at 10:29 PM, M. Manna <manme...@gmail.com> wrote: > > > > > Hi, > > > > > > What's the status of your SSL? Have you verified that the setup is > > working? > > > > > > You can enable rough logins using log4j.properties file supplier with > > kafka > > > and set the root logging level to DEBUG. This prints out more info to > > trace > > > things. Also, you can enable security logging by adding > > > -Djavax.security.debug=all > > > > > > Please share your producer/broker configs with us. > > > > > > Kindest Regards, > > > M. Manna > > > > > > On 9 August 2017 at 14:38, Ascot Moss <ascot.m...@gmail.com> wrote: > > > > > > > Hi, > > > > > > > > > > > > I have setup Kafka 0.10.2.1 with SSL. > > > > > > > > > > > > Check Status: > > > > > > > > openssl s_client -debug -connect n1:9093 -tls1 > > > > > > > > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA > > > > > > > > ... SSL-Session: > > > > > > > > Protocol : TLSv1 > > > > > > > > PSK identity hint: None > > > > > > > > Start Time: 1502285690 > > > > > > > > Timeout : 7200 (sec) > > > > > > > > Verify return code: 19 (self signed certificate in certificate > > chain) > > > > > > > > > > > > Create Topic: > > > > > > > > kafka-topics.sh --create --zookeeper n1:2181,n2:2181,n3:2181 > > > > --replication-factor 3 --partitions 3 --topic test02 > > > > > > > > ERROR [ReplicaFetcherThread-2-111], Error for partition [test02,2] to > > > > broker 1:org.apache.kafka.common.errors. > UnknownTopicOrPartitionExcepti > > > on: > > > > This server does not host this topic-partition. > > > > (kafka.server.ReplicaFetcherThread) > > > > > > > > However, if I run describe topic, I can see it is created > > > > > > > > > > > > > > > > Describe Topic: > > > > > > > > kafka-topics.sh --zookeeper n1:2181,n2:2181,n3:2181 --describe > --topic > > > > test02 > > > > > > > > Topic:test02 PartitionCount:3 ReplicationFactor:3 Configs: > > > > > > > > Topic: test02 Partition: 0 Leader: 12 Replicas: 12,13,11 Isr: > 12,13,11 > > > > > > > > Topic: test02 Partition: 1 Leader: 13 Replicas: 13,11,12 Isr: > 13,11,12 > > > > > > > > Topic: test02 Partition: 2 Leader: 11 Replicas: 11,12,13 Isr: > 11,12,13 > > > > > > > > > > > > Consumer: > > > > > > > > kafka-console-consumer.sh --bootstrap-server n1:9093 > --consumer.config > > > > /home/kafka/config/consumer.n1.properties --topic test02 > > > --from-beginning > > > > > > > > > > > > > > > > Producer: > > > > > > > > kafka-console-producer.sh --broker-list n1:9093 --producer.config > > > > /homey/kafka/config/producer.n1.properties --sync --topic test02 > > > > > > > > ERROR Error when sending message to topic test02 with key: null, > > value: 0 > > > > bytes with error: > > > > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) > > > > > > > > org.apache.kafka.common.errors.TimeoutException: Expiring 1 > record(s) > > > for > > > > test02-1: 1506 ms has passed since batch creation plus linger time > > > > > > > > > > > > How to resolve it? > > > > > > > > Regards > > > > > > > > > >