Hi Kafka Users,
I am trying to setup a Simple Authentication Mechanism for my Kafka
Instance running on my Virtual Box VM.
I am facing a lot of difficulty in starting the Zookeeper 3.4.10.
The scenario is like this...
I have a single admin User called sharjosh who's starting both the
Zookeeper and Kafka processes on the VM.
I tried to use the following command to start the zookeeper
~/zookeeper-3.4.10$ export
JVMFLAGS='-Djava.security.auth.login.config=/home/sharjosh/kafka_2.10-0.10.2.1/config/zk_server_jaas.conf';
bin/zkServer.sh start-foreground conf/zoo.cfg
and get
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
at
org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:116)
at
org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:53)
at
org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:87)
at
org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:117)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:82)
at
org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:200)java.io.IOException:
No JAAS configuration section named 'Server' was foundin
'/home/sharjosh/kafka_2.10-0.10.2.1/config/zk_server_jaas.conf'.2017-06-04
22:43:31,324 [myid:] - ERROR [main:ZooKeeperServerMain@64] - Unexpected
exception, exiting abnormally2017-06-04 22:43:31,323 [myid:] - ERROR
[main:ServerCnxnFactory@199] - No JAAS configuration section named 'Server'
was foundin '/home/sharjosh/kafka_2.10-0.10.2.1/config/zk_server_jaas.conf'.
My JAAS Config file for Zookeeper is
Server {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
useTicketCache=true
keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
principal="sharjosh@localhost"
};
My zoo.cfg file is
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/home/sharjosh/zookeeper-3.4.10/data
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
maxClientCnxns=500
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
autopurge.purgeInterval=1
#
https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000
My Kafka startup command is
export KAFKA_OPTS='-Djava.security.krb5.conf=/etc/krb5.conf
-Djava.security.auth.login.config=/home/sharjosh/kafka_2.10-0.10.2.1/config/kafka_server_jaas.conf
-Dzookeeper.sasl.client=zKClient'; ./kafka-server-start.sh
/home/sharjosh/kafka-0.10.2.1-src/config/server.properties
My Kafka server's JAAS file is
KafkaServer {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
serviceName=kafka
useTicketCache=true
keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
principal="sharjosh@localhost"
};
zKClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
serviceName=zookeeper
useTicketCache=true
keyTab="/home/sharjosh/kafkaSecurity/sharjosh.keytab"
principal="sharjosh@localhost"
};
My security settings in kafka's server.properties
listeners=SASL_PLAINTEXT://:9092
advertised.listeners=SASL_PLAINTEXT://:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=GSSAPI,PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
zookeeper.connect=localhost:2181
# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000
On the other hand in Kafka Broker startup also I get errors as there's no
Zookeeper available.
org.apache.kafka.common.KafkaException: Exception while loading Zookeeper
JAAS login context 'Client'
at
org.apache.kafka.common.security.JaasUtils.isZkSecurityEnabled(JaasUtils.java:154)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:310)
at kafka.server.KafkaServer.startup(KafkaServer.scala:187)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
Caused by: java.lang.SecurityException: java.io.IOException: Configuration
Error:
Line 9: expected [option key]
at sun.security.provider.ConfigFile$Spi.<init>(ConfigFile.java:137)
at sun.security.provider.ConfigFile.<init>(ConfigFile.java:102)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at java.lang.Class.newInstance(Class.java:442)
at javax.security.auth.login.Configuration$2.run(Configuration.java:255)
at javax.security.auth.login.Configuration$2.run(Configuration.java:247)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.Configuration.getConfiguration(Configuration.java:246)
at
org.apache.kafka.common.security.JaasUtils.isZkSecurityEnabled(JaasUtils.java:151)
... 5 more
Caused by: java.io.IOException: Configuration Error:
Line 9: expected [option key]
...
So please advice me on what I am missing in the Configs to properly start
Zookeeper to move forward. I have put in many days of investigation leading
to pretty much the same result.
I am eagerly hoping the group and its users might have already conquered it
and would help me. Waiting to hear back from you.
Thanks a lot,
Sharat
