if you remove host.name, advertised.host.name and port from
server.properties, does it work for you?

I am using SSL without ACL. it seems to be working fine.

On 9 August 2017 at 22:03, Ascot Moss <ascot.m...@gmail.com> wrote:

> About:
> zookeeper-shell.sh localhost:2181
> get /brokers/ids/11
>
>
> The result:
>
> zookeeper-shell.sh n1.test.com:2181
>
> Connecting to n1.test.com:2181
>
> Welcome to ZooKeeper!
>
> JLine support is disabled
>
> WATCHER::
>
> WatchedEvent state:SyncConnected type:None path:null
>
> WATCHER::
>
>
>
>
> get /brokers/ids/11
>
> WatchedEvent state:SaslAuthenticated type:None path:null
>
> {"listener_security_protocol_map":{"SSL":"SSL"},"endpoints":["SSL://
> n1.test.com:9093
> "],"jmx_port":-1,"host":null,"timestamp":"1502310695312","
> port":-1,"version":4}
>
> cZxid = 0x40002787d
>
> ctime = Thu Aug 10 04:31:37 HKT 2017
>
> mZxid = 0x40002787d
>
> mtime = Thu Aug 10 04:31:37 HKT 2017
>
> pZxid = 0x40002787d
>
> cversion = 0
>
> dataVersion = 0
>
> aclVersion = 0
>
> ephemeralOwner = 0x35d885c689c00a6
>
> dataLength = 168
>
> numChildren = 0
>
> On Thu, Aug 10, 2017 at 4:46 AM, Ascot Moss <ascot.m...@gmail.com> wrote:
>
> > About:  zookeeper-shell.sh localhost:2181
> > get /brokers/ids/11
> >
> > The result:
> >
> > zookeeper-shell.sh n1.test.com:2181
> >
> > Connecting to n1.test.com:2181
> >
> > Welcome to ZooKeeper!
> >
> > JLine support is disabled
> >
> > WATCHER::
> >
> > WatchedEvent state:SyncConnected type:None path:null
> >
> > WATCHER::
> >
> > WatchedEvent state:SaslAuthenticated type:None path:null
> >
> >
> > On Thu, Aug 10, 2017 at 4:43 AM, Ascot Moss <ascot.m...@gmail.com>
> wrote:
> >
> >> FYI, about zookeeper, I used my existing zookeeper (as I have existing
> >> zookeeper up and running, which is also used for hbase)
> >>
> >> zookeeper versoom: 3.4.10
> >>
> >> zoo.cfg
> >> ######
> >>
> >> tickTime=2000
> >>
> >> initLimit=10
> >>
> >> syncLimit=5
> >>
> >> dataDir=/usr/local/zookeeper/data
> >>
> >> dataLogDir=/usr/local/zookeeper/datalog
> >>
> >> clientPort=2181
> >>
> >> maxClientCnxns=60
> >>
> >> server.1=n1.test.com:2888:3888
> >>
> >> server.2=n2.test.com:2888:3888
> >>
> >> server.3=n3.test.com:2888:3888
> >>
> >> authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenti
> >> cationProvider
> >>
> >> jaasLoginRenew=3600000
> >>
> >> requireClientAuthScheme=sasl
> >>
> >> zookeeper.allowSaslFailedClients=false
> >>
> >> kerberos.removeHostFromPrincipal=true
> >>
> >> ######
> >>
> >>
> >>
> >> On Thu, Aug 10, 2017 at 4:35 AM, Ascot Moss <ascot.m...@gmail.com>
> wrote:
> >>
> >>> server.properties
> >>>
> >>> ######
> >>>
> >>> broker.id=11
> >>>
> >>> port=9093
> >>>
> >>> host.name=n1
> >>>
> >>> advertised.host.name=192.168.0.11
> >>>
> >>> allow.everyone.if.no.acl.found=true
> >>>
> >>> super.users=User:CN=n1.test.com,OU=TEST,O=TEST,L=TEST,ST=TEST,C=TEST
> >>>
> >>> listeners=SSL://n1.test.com:9093 <http://n1.test.com:9092/>
> >>>
> >>> advertised.listeners=SSL://n1.test.com:9093 <http://n1.test.com:9092/>
> >>>
> >>> ssl.client.auth=required
> >>>
> >>> ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
> >>>
> >>> ssl.keystore.type=JKS
> >>>
> >>> ssl.truststore.type=JKS
> >>>
> >>> security.inter.broker.protocol=SSL
> >>>
> >>> ssl.keystore.location=/home/kafka/kafka.server.keystore.jks
> >>>
> >>> ssl.keystore.password=Test2017
> >>>
> >>> ssl.key.password=Test2017
> >>>
> >>> ssl.truststore.location=/home/kafka/kafka.server.truststore.jks
> >>>
> >>> ssl.truststore.password=Test2017
> >>>
> >>> authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
> >>>
> >>> principal.builder.class=org.apache.kafka.common.security.aut
> >>> h.DefaultPrincipalBuilder
> >>>
> >>> num.replica.fetchers=4
> >>>
> >>> replica.fetch.max.bytes=1048576
> >>>
> >>> replica.fetch.wait.max.ms=500
> >>>
> >>> replica.high.watermark.checkpoint.interval.ms=5000
> >>>
> >>> replica.socket.timeout.ms=30000
> >>>
> >>> replica.socket.receive.buffer.bytes=65536
> >>>
> >>> replica.lag.time.max.ms=10000
> >>>
> >>> controller.socket.timeout.ms=30000
> >>>
> >>> controller.message.queue.size=10
> >>>
> >>> default.replication.factor=3
> >>>
> >>> log.dirs=/usr/log/kafka
> >>>
> >>> kafka.logs.dir=/usr/log/kafka
> >>>
> >>> num.partitions=20
> >>>
> >>> message.max.bytes=1000000
> >>>
> >>> auto.create.topics.enable=true
> >>>
> >>> log.index.interval.bytes=4096
> >>>
> >>> log.index.size.max.bytes=10485760
> >>>
> >>> log.retention.hours=720
> >>>
> >>> log.flush.interval.ms=10000
> >>>
> >>> log.flush.interval.messages=20000
> >>>
> >>> log.flush.scheduler.interval.ms=2000
> >>>
> >>> log.roll.hours=168
> >>>
> >>> log.retention.check.interval.ms=300000
> >>>
> >>> log.segment.bytes=1073741824
> >>>
> >>> delete.topic.enable=true
> >>>
> >>> socket.request.max.bytes=104857600
> >>>
> >>> socket.receive.buffer.bytes=1048576
> >>>
> >>> socket.send.buffer.bytes=1048576
> >>>
> >>> num.io.threads=8
> >>>
> >>> num.network.threads=8
> >>>
> >>> queued.max.requests=16
> >>>
> >>> fetch.purgatory.purge.interval.requests=100
> >>>
> >>> producer.purgatory.purge.interval.requests=100
> >>>
> >>> zookeeper.connect=n1:2181,n2:2181,n3:2181
> >>>
> >>> zookeeper.connection.timeout.ms=2000
> >>>
> >>> zookeeper.sync.time.ms=2000
> >>>
> >>> ######
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> producer.properties
> >>>
> >>> ######
> >>>
> >>> bootstrap.servers=n1.test.com:9093 <http://n1.test.com:9092/>
> >>>
> >>> security.protocol=SSL
> >>>
> >>> ssl.truststore.location=/home/kafka/kafka.client.truststore.jks
> >>>
> >>> ssl.truststore.password=testkafka
> >>>
> >>> ssl.keystore.location=/home/kafka/kafka.client.keystore.jks
> >>>
> >>> ssl.keystore.password=testkafka
> >>>
> >>> ssl.key.password=testkafka
> >>> #####
> >>>
> >>>
> >>> (I had tried to switch to another port, 9093 is the correct port)
> >>>
> >>> On Thu, Aug 10, 2017 at 4:28 AM, M. Manna <manme...@gmail.com> wrote:
> >>>
> >>>> Your openssl test is showing connected with port 9092. but your
> previous
> >>>> messages show 9093 - is there some typo issues? Where is SSL running
> >>>>
> >>>> Please share the following and don't leave any details out. This will
> >>>> only
> >>>> create more assumptions.
> >>>>
> >>>> 1) server.properties
> >>>> 2) Zookeeper.properties
> >>>>
> >>>> Also, run the following command (when the cluster is running)
> >>>> zookeeper-shell.sh localhost:2181
> >>>> get /brokers/ids/11
> >>>>
> >>>> Does it show that your broker #11 is connected?
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On 9 August 2017 at 21:17, Ascot Moss <ascot.m...@gmail.com> wrote:
> >>>>
> >>>> > Dear Manna,
> >>>> >
> >>>> >
> >>>> > What's the status of your SSL? Have you verified that the setup is
> >>>> working?
> >>>> > Yes, I used "
> >>>> >
> >>>> > openssl s_client -debug -connect n1.test.com:9092 -tls1
> >>>> > Output:
> >>>> >
> >>>> > CONNECTED(00000003)
> >>>> >
> >>>> > write to 0x853e70 [0x89fd43] (155 bytes => 155 (0x9B))
> >>>> >
> >>>> > 0000 - 16 03 01 00 96 01 00 00-92 03 01 59 8b 6d 0d b1
> >>>>  ...........Y.m..
> >>>> > ...
> >>>> >
> >>>> > Server certificate
> >>>> >
> >>>> > -----BEGIN CERTIFICATE-----
> >>>> >
> >>>> > CwwCSEsxGT............
> >>>> >
> >>>> > -----END CERTIFICATE-----
> >>>> >
> >>>> > ---
> >>>> >
> >>>> > SSL handshake has read 2470 bytes and written 161 bytes
> >>>> >
> >>>> > ---
> >>>> >
> >>>> > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
> >>>> >
> >>>> >     PSK identity hint: None
> >>>> >
> >>>> >     Start Time: 1502309645
> >>>> >
> >>>> >     Timeout   : 7200 (sec)
> >>>> >
> >>>> >     Verify return code: 19 (self signed certificate in certificate
> >>>> chain)
> >>>> >
> >>>> > ---
> >>>> >
> >>>> > Regards
> >>>> >
> >>>> > On Wed, Aug 9, 2017 at 10:29 PM, M. Manna <manme...@gmail.com>
> wrote:
> >>>> >
> >>>> > > Hi,
> >>>> > >
> >>>> > > What's the status of your SSL? Have you verified that the setup is
> >>>> > working?
> >>>> > >
> >>>> > > You can enable rough logins using log4j.properties file supplier
> >>>> with
> >>>> > kafka
> >>>> > > and set the root logging level to DEBUG. This prints out more info
> >>>> to
> >>>> > trace
> >>>> > > things. Also, you can enable security logging by adding
> >>>> > > -Djavax.security.debug=all
> >>>> > >
> >>>> > > Please share your producer/broker configs with us.
> >>>> > >
> >>>> > > Kindest Regards,
> >>>> > > M. Manna
> >>>> > >
> >>>> > > On 9 August 2017 at 14:38, Ascot Moss <ascot.m...@gmail.com>
> wrote:
> >>>> > >
> >>>> > > > Hi,
> >>>> > > >
> >>>> > > >
> >>>> > > > I have setup Kafka 0.10.2.1 with SSL.
> >>>> > > >
> >>>> > > >
> >>>> > > > Check Status:
> >>>> > > >
> >>>> > > > openssl s_client -debug -connect n1:9093 -tls1
> >>>> > > >
> >>>> > > > New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
> >>>> > > >
> >>>> > > > ... SSL-Session:
> >>>> > > >
> >>>> > > >     Protocol  : TLSv1
> >>>> > > >
> >>>> > > >     PSK identity hint: None
> >>>> > > >
> >>>> > > >     Start Time: 1502285690
> >>>> > > >
> >>>> > > >     Timeout   : 7200 (sec)
> >>>> > > >
> >>>> > > >     Verify return code: 19 (self signed certificate in
> certificate
> >>>> > chain)
> >>>> > > >
> >>>> > > >
> >>>> > > > Create Topic:
> >>>> > > >
> >>>> > > > kafka-topics.sh --create --zookeeper n1:2181,n2:2181,n3:2181
> >>>> > > > --replication-factor 3 --partitions 3 --topic test02
> >>>> > > >
> >>>> > > > ERROR [ReplicaFetcherThread-2-111], Error for partition
> >>>> [test02,2] to
> >>>> > > > broker 1:org.apache.kafka.common.erro
> >>>> rs.UnknownTopicOrPartitionExcepti
> >>>> > > on:
> >>>> > > > This server does not host this topic-partition.
> >>>> > > > (kafka.server.ReplicaFetcherThread)
> >>>> > > >
> >>>> > > > However, if I run describe topic, I can see it is created
> >>>> > > >
> >>>> > > >
> >>>> > > >
> >>>> > > > Describe Topic:
> >>>> > > >
> >>>> > > > kafka-topics.sh --zookeeper n1:2181,n2:2181,n3:2181 --describe
> >>>> --topic
> >>>> > > > test02
> >>>> > > >
> >>>> > > > Topic:test02 PartitionCount:3 ReplicationFactor:3 Configs:
> >>>> > > >
> >>>> > > > Topic: test02 Partition: 0 Leader: 12 Replicas: 12,13,11 Isr:
> >>>> 12,13,11
> >>>> > > >
> >>>> > > > Topic: test02 Partition: 1 Leader: 13 Replicas: 13,11,12 Isr:
> >>>> 13,11,12
> >>>> > > >
> >>>> > > > Topic: test02 Partition: 2 Leader: 11 Replicas: 11,12,13 Isr:
> >>>> 11,12,13
> >>>> > > >
> >>>> > > >
> >>>> > > > Consumer:
> >>>> > > >
> >>>> > > > kafka-console-consumer.sh --bootstrap-server n1:9093
> >>>> --consumer.config
> >>>> > > > /home/kafka/config/consumer.n1.properties --topic test02
> >>>> > > --from-beginning
> >>>> > > >
> >>>> > > >
> >>>> > > >
> >>>> > > > Producer:
> >>>> > > >
> >>>> > > > kafka-console-producer.sh --broker-list n1:9093
> --producer.config
> >>>> > > > /homey/kafka/config/producer.n1.properties --sync --topic
> test02
> >>>> > > >
> >>>> > > > ERROR Error when sending message to topic test02 with key: null,
> >>>> > value: 0
> >>>> > > > bytes with error:
> >>>> > > > (org.apache.kafka.clients.producer.internals.ErrorLoggingCal
> >>>> lback)
> >>>> > > >
> >>>> > > > org.apache.kafka.common.errors.TimeoutException: Expiring 1
> >>>> record(s)
> >>>> > > for
> >>>> > > > test02-1: 1506 ms has passed since batch creation plus linger
> time
> >>>> > > >
> >>>> > > >
> >>>> > > > How to resolve it?
> >>>> > > >
> >>>> > > > Regards
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>

Reply via email to