Kafka ACLs CLI Auth Error

[Derar Alassi]

Hi all,

I have  3-node ZK and Kafka clusters. I have secured ZK with SASL. I got
the keytabs done for my brokers and they can connect to the ZK ensemble
just fine with no issues. All gravy!

Now, I am trying to set ACLs using the kafka-acls.sh CLI. Before that, I
did export the KAFKA_OPTS using the following command:


 export  
KAFKA_OPTS="-Djava.security.auth.login.config=<path>/kafka_server_jaas.conf
-Djavax.net.debug=all -Dsun.security.krb5.debug=true -Djavax.net.debug=all
-Dsun.security.krb5.debug=true -Djava.security.krb5.conf=<path to krb
conf>/krb5.conf"

I enabled extra debugging too. The JAAS file has the following info:

KafkaServer {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    storeKey=true
    keyTab="/etc/<hostname>+kafka.keytab"
    principal="kafka/<hostname>@MY_DOMAIN";
};
Client {
    com.sun.security.auth.module.Krb5LoginModule required
    useKeyTab=true
    useTicketCache=true
    storeKey=true
    keyTab="/etc/<hostname>+kafka.keytab"
    principal="kafka/<hostname>@MY_DOMAIN";
};

Note that I enabled useTicketCache in the client section.

I know that my krb5.conf file is good since the brokers are healthy and
consumer/producers are able to do their work.

Two scenarios:

1. When I enabled the useTicketCache=true, I get the following error:

*Aug 08, 2016 8:42:46 PM org.apache.zookeeper.ClientCnxn$SendThread
startConnectWARNING: SASL configuration failed:
javax.security.auth.login.LoginException: No key to store Will continue
connection to Zookeeper server without SASL authentication, if Zookeeper
server allows it.*

I execute "kinit kafka/<hostname>@<MY_DOMAIN> -k -t
/etc/<hostname>+kafka.keytab " on the same shell where I run the .sh CLI
tool.
2. When I remove userTicketCache, I get the following error:








*Aug 08, 2016 9:03:38 PM org.apache.zookeeper.ZooKeeper closeINFO: Session:
0x356621f18f70009 closedError while executing ACL command:
org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
NoAuth for /kafka-acl/TopicAug 08, 2016 9:03:38 PM
org.apache.zookeeper.ClientCnxn$EventThread runINFO: EventThread shut
downorg.I0Itec.zkclient.exception.ZkException:
org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
NoAuth for /kafka-acl/Topic        at
org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:68)*


Here is the command I run to set the ACLs in all cases:
./bin/kafka-acls.sh --authorizer-properties zookeeper.connect=<zk-host>:2181
--add --allow-principal User:Bob --producer --topic ssl-topic


I use Kafka 0.9.0.1. Note that I am using the same keytabs that my Brokers
(Kafka services) are using.


Any ideas what I am doing wrong or what I should do differently to get ACLs
set?

Thanks,
Derar