Re: ZooKeeper in secure mode
For testing it on the real Hadoop cluster with real Kerberos, we used ZooKeeper 3.5.5. AFAIK ZooKeeper 3.5.6 should behave just the same in terms of SASL and SSL. (I also created unit tests for SASL + SSL in the PRs of the Jira ticket I mentioned, those can give you configuration examples for the branches 3.5 and master) Regards, Mate On Fri, Jan 17, 2020 at 4:40 AM Praveen Kumar K S wrote: > Thanks Mate. May I know the version of zookeeper you are using? > > Regards, > Praveen Kumar K S > +91-9986855625 > > > On Thu, Jan 16, 2020 at 8:45 PM Szalay-Bekő Máté < > szalay.beko.m...@gmail.com> > wrote: > > > Hi Praveen, > > > > Regarding SASL, some useful links: > > - > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication > > (I > > just updated this page today) > > - I was also checking the Kerberos JAAS configs when I tried these things > > locally: > > > > > https://docs.oracle.com/javase/8/docs/jre/api/security/jaas/spec/com/sun/security/auth/module/Krb5LoginModule.html > > - this is a good howto as well: > https://github.com/ekoontz/zookeeper/wiki > > - > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Server-Server+mutual+authentication > > > > In this Jira case you can see some zoo.cfg and client configs that we > used > > to test SASL + SSL: > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-3482?focusedCommentId=16998033=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16998033 > > > > With these configs we were managed to use ZooKeeper SASL + SSL on a real > > Hadoop cluster using MIT Kerberos. > > > > Mate > > > > On Thu, Jan 16, 2020 at 10:39 AM Praveen Kumar K S < > > prav...@securelyshare.com> wrote: > > > > > Thanks Enrico. I was also looking at > > > https://issues.apache.org/jira/browse/ZOOKEEPER-2220 who is facing > same > > > issue. > > > > > > I will try with your suggestion. My requirement is to enable SASL based > > > authentication between server-server and client-server. > > > > > > Please advise if I'm looking at the right place or is there any better > > > documentation. > > > > > > Regards, > > > Praveen Kumar K S > > > +91-9986855625 > > > > > > > > > On Thu, Jan 16, 2020 at 3:01 PM Enrico Olivelli - Diennea < > > > enrico.olive...@diennea.com> wrote: > > > > > > > Praveen > > > > In order to use Netty it is better for you to use 3.5.6 that contains > > > > Netty 4, ZooKeeper 3.4.x uses the deprecated Netty 3. For TSL, and it > > is > > > > known to have security flaws and it is no more maintained > > > > > > > > Btw your problem looks like there is a missing class and it is weird > > > > > > > > Enrico > > > > > > > > Il giorno 16/01/20, 10:25 "Praveen Kumar K S" < > > > prav...@securelyshare.com> > > > > ha scritto: > > > > > > > > Hello, > > > > > > > > I'm looking for help on enabling authentication in zookeeper. > > Please > > > > note > > > > below approach I have tried. > > > > > > > > 1. I followed > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > > > > 2. I'm deploying zookeeper as single node using docker > > > > 3. Zookeeper version is 3.4.13 > > > > 4. Below are some important environmental variables in zookeeper > > > > container > > > > > > > > > > > > > > > > > > CLIENT_JVMFLAGS=-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > > > -Dzookeeper.client.secure=true > > > > > > > > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > > > > -Dzookeeper.ssl.keyStore.password=XX@123 > > > > > > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > > > > -Dzookeeper.ssl.trustStore.password=XX@123 > > > > > > > > > > > > > > > > > > SERVER_JVMFLAGS=-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > > > > > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > > > > -Dzookeeper.ssl.keyStore.password=XX@123 > > > > > > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > > > > -Dzookeeper.ssl.trustStore.password=XX@123 > > > > > > > > > > > > > > > > > > zookeeper.serverCnxnFactory="org.apache.zookeeper.server.NettyServerCnxnFactory" > > > > > > > > 5. Below is conf file > > > > server.1=0.0.0.0:2888:3888 > > > > secureClientPort=2281 > > > > initLimit=5 > > > > syncLimit=2 > > > > tickTime=2000 > > > > clientPort=2181 > > > > clientPortAddress=zookeeper > > > > dataLogDir=/opt/vault/zookeeper/logs > > > > dataDir=/opt/vault/zookeeper/data > > > > > > > > 6. Zookeeper is healthy > > > > 7. I tried connecting to Zookeeper server from my machine using > > > > zkCli.sh. > > > > But getting below error > > > > > > > > 2020-01-16 14:21:27,798 [myid:] - INFO [main:ZooKeeper@442] - > > > > Initiating > > > > client connection,
Re: ZooKeeper in secure mode
Thanks Mate. May I know the version of zookeeper you are using? Regards, Praveen Kumar K S +91-9986855625 On Thu, Jan 16, 2020 at 8:45 PM Szalay-Bekő Máté wrote: > Hi Praveen, > > Regarding SASL, some useful links: > - > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication > (I > just updated this page today) > - I was also checking the Kerberos JAAS configs when I tried these things > locally: > > https://docs.oracle.com/javase/8/docs/jre/api/security/jaas/spec/com/sun/security/auth/module/Krb5LoginModule.html > - this is a good howto as well: https://github.com/ekoontz/zookeeper/wiki > - > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Server-Server+mutual+authentication > > In this Jira case you can see some zoo.cfg and client configs that we used > to test SASL + SSL: > > https://issues.apache.org/jira/browse/ZOOKEEPER-3482?focusedCommentId=16998033=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16998033 > > With these configs we were managed to use ZooKeeper SASL + SSL on a real > Hadoop cluster using MIT Kerberos. > > Mate > > On Thu, Jan 16, 2020 at 10:39 AM Praveen Kumar K S < > prav...@securelyshare.com> wrote: > > > Thanks Enrico. I was also looking at > > https://issues.apache.org/jira/browse/ZOOKEEPER-2220 who is facing same > > issue. > > > > I will try with your suggestion. My requirement is to enable SASL based > > authentication between server-server and client-server. > > > > Please advise if I'm looking at the right place or is there any better > > documentation. > > > > Regards, > > Praveen Kumar K S > > +91-9986855625 > > > > > > On Thu, Jan 16, 2020 at 3:01 PM Enrico Olivelli - Diennea < > > enrico.olive...@diennea.com> wrote: > > > > > Praveen > > > In order to use Netty it is better for you to use 3.5.6 that contains > > > Netty 4, ZooKeeper 3.4.x uses the deprecated Netty 3. For TSL, and it > is > > > known to have security flaws and it is no more maintained > > > > > > Btw your problem looks like there is a missing class and it is weird > > > > > > Enrico > > > > > > Il giorno 16/01/20, 10:25 "Praveen Kumar K S" < > > prav...@securelyshare.com> > > > ha scritto: > > > > > > Hello, > > > > > > I'm looking for help on enabling authentication in zookeeper. > Please > > > note > > > below approach I have tried. > > > > > > 1. I followed > > > > > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > > > 2. I'm deploying zookeeper as single node using docker > > > 3. Zookeeper version is 3.4.13 > > > 4. Below are some important environmental variables in zookeeper > > > container > > > > > > > > > > > > CLIENT_JVMFLAGS=-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > > -Dzookeeper.client.secure=true > > > > > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > > > -Dzookeeper.ssl.keyStore.password=XX@123 > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > > > -Dzookeeper.ssl.trustStore.password=XX@123 > > > > > > > > > > > > SERVER_JVMFLAGS=-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > > > -Dzookeeper.ssl.keyStore.password=XX@123 > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > > > -Dzookeeper.ssl.trustStore.password=XX@123 > > > > > > > > > > > > zookeeper.serverCnxnFactory="org.apache.zookeeper.server.NettyServerCnxnFactory" > > > > > > 5. Below is conf file > > > server.1=0.0.0.0:2888:3888 > > > secureClientPort=2281 > > > initLimit=5 > > > syncLimit=2 > > > tickTime=2000 > > > clientPort=2181 > > > clientPortAddress=zookeeper > > > dataLogDir=/opt/vault/zookeeper/logs > > > dataDir=/opt/vault/zookeeper/data > > > > > > 6. Zookeeper is healthy > > > 7. I tried connecting to Zookeeper server from my machine using > > > zkCli.sh. > > > But getting below error > > > > > > 2020-01-16 14:21:27,798 [myid:] - INFO [main:ZooKeeper@442] - > > > Initiating > > > client connection, connectString=zookeeper:2281 > sessionTimeout=3 > > > watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@531d72ca > > > Exception in thread "main" java.io.IOException: Couldn't > instantiate > > > org.apache.zookeeper.ClientCnxnSocketNetty > > > at > > > org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1851) > > > at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:453) > > > at > > > org.apache.zookeeper.ZooKeeperMain.connectToZK(ZooKeeperMain.java:283) > > > at > org.apache.zookeeper.ZooKeeperMain.(ZooKeeperMain.java:297) > > > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:290) > > > Caused by: java.lang.ClassNotFoundException: > > >
Re: ZooKeeper in secure mode
Hi Praveen, Regarding SASL, some useful links: - https://cwiki.apache.org/confluence/display/ZOOKEEPER/Client-Server+mutual+authentication (I just updated this page today) - I was also checking the Kerberos JAAS configs when I tried these things locally: https://docs.oracle.com/javase/8/docs/jre/api/security/jaas/spec/com/sun/security/auth/module/Krb5LoginModule.html - this is a good howto as well: https://github.com/ekoontz/zookeeper/wiki - https://cwiki.apache.org/confluence/display/ZOOKEEPER/Server-Server+mutual+authentication In this Jira case you can see some zoo.cfg and client configs that we used to test SASL + SSL: https://issues.apache.org/jira/browse/ZOOKEEPER-3482?focusedCommentId=16998033=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16998033 With these configs we were managed to use ZooKeeper SASL + SSL on a real Hadoop cluster using MIT Kerberos. Mate On Thu, Jan 16, 2020 at 10:39 AM Praveen Kumar K S < prav...@securelyshare.com> wrote: > Thanks Enrico. I was also looking at > https://issues.apache.org/jira/browse/ZOOKEEPER-2220 who is facing same > issue. > > I will try with your suggestion. My requirement is to enable SASL based > authentication between server-server and client-server. > > Please advise if I'm looking at the right place or is there any better > documentation. > > Regards, > Praveen Kumar K S > +91-9986855625 > > > On Thu, Jan 16, 2020 at 3:01 PM Enrico Olivelli - Diennea < > enrico.olive...@diennea.com> wrote: > > > Praveen > > In order to use Netty it is better for you to use 3.5.6 that contains > > Netty 4, ZooKeeper 3.4.x uses the deprecated Netty 3. For TSL, and it is > > known to have security flaws and it is no more maintained > > > > Btw your problem looks like there is a missing class and it is weird > > > > Enrico > > > > Il giorno 16/01/20, 10:25 "Praveen Kumar K S" < > prav...@securelyshare.com> > > ha scritto: > > > > Hello, > > > > I'm looking for help on enabling authentication in zookeeper. Please > > note > > below approach I have tried. > > > > 1. I followed > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > > 2. I'm deploying zookeeper as single node using docker > > 3. Zookeeper version is 3.4.13 > > 4. Below are some important environmental variables in zookeeper > > container > > > > > > > CLIENT_JVMFLAGS=-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > -Dzookeeper.client.secure=true > > > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > > -Dzookeeper.ssl.keyStore.password=XX@123 > > > > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > > -Dzookeeper.ssl.trustStore.password=XX@123 > > > > > > > SERVER_JVMFLAGS=-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > > -Dzookeeper.ssl.keyStore.password=XX@123 > > > > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > > -Dzookeeper.ssl.trustStore.password=XX@123 > > > > > > > zookeeper.serverCnxnFactory="org.apache.zookeeper.server.NettyServerCnxnFactory" > > > > 5. Below is conf file > > server.1=0.0.0.0:2888:3888 > > secureClientPort=2281 > > initLimit=5 > > syncLimit=2 > > tickTime=2000 > > clientPort=2181 > > clientPortAddress=zookeeper > > dataLogDir=/opt/vault/zookeeper/logs > > dataDir=/opt/vault/zookeeper/data > > > > 6. Zookeeper is healthy > > 7. I tried connecting to Zookeeper server from my machine using > > zkCli.sh. > > But getting below error > > > > 2020-01-16 14:21:27,798 [myid:] - INFO [main:ZooKeeper@442] - > > Initiating > > client connection, connectString=zookeeper:2281 sessionTimeout=3 > > watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@531d72ca > > Exception in thread "main" java.io.IOException: Couldn't instantiate > > org.apache.zookeeper.ClientCnxnSocketNetty > > at > > org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1851) > > at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:453) > > at > > org.apache.zookeeper.ZooKeeperMain.connectToZK(ZooKeeperMain.java:283) > > at org.apache.zookeeper.ZooKeeperMain.(ZooKeeperMain.java:297) > > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:290) > > Caused by: java.lang.ClassNotFoundException: > > org.apache.zookeeper.ClientCnxnSocketNetty > > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > > at java.lang.Class.forName0(Native Method) > > at java.lang.Class.forName(Class.java:264) > > at > >
Re: ZooKeeper in secure mode
Thanks Enrico. I was also looking at https://issues.apache.org/jira/browse/ZOOKEEPER-2220 who is facing same issue. I will try with your suggestion. My requirement is to enable SASL based authentication between server-server and client-server. Please advise if I'm looking at the right place or is there any better documentation. Regards, Praveen Kumar K S +91-9986855625 On Thu, Jan 16, 2020 at 3:01 PM Enrico Olivelli - Diennea < enrico.olive...@diennea.com> wrote: > Praveen > In order to use Netty it is better for you to use 3.5.6 that contains > Netty 4, ZooKeeper 3.4.x uses the deprecated Netty 3. For TSL, and it is > known to have security flaws and it is no more maintained > > Btw your problem looks like there is a missing class and it is weird > > Enrico > > Il giorno 16/01/20, 10:25 "Praveen Kumar K S" > ha scritto: > > Hello, > > I'm looking for help on enabling authentication in zookeeper. Please > note > below approach I have tried. > > 1. I followed > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > 2. I'm deploying zookeeper as single node using docker > 3. Zookeeper version is 3.4.13 > 4. Below are some important environmental variables in zookeeper > container > > > CLIENT_JVMFLAGS=-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > -Dzookeeper.client.secure=true > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > -Dzookeeper.ssl.keyStore.password=XX@123 > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > -Dzookeeper.ssl.trustStore.password=XX@123 > > > SERVER_JVMFLAGS=-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks > -Dzookeeper.ssl.keyStore.password=XX@123 > > -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks > -Dzookeeper.ssl.trustStore.password=XX@123 > > > zookeeper.serverCnxnFactory="org.apache.zookeeper.server.NettyServerCnxnFactory" > > 5. Below is conf file > server.1=0.0.0.0:2888:3888 > secureClientPort=2281 > initLimit=5 > syncLimit=2 > tickTime=2000 > clientPort=2181 > clientPortAddress=zookeeper > dataLogDir=/opt/vault/zookeeper/logs > dataDir=/opt/vault/zookeeper/data > > 6. Zookeeper is healthy > 7. I tried connecting to Zookeeper server from my machine using > zkCli.sh. > But getting below error > > 2020-01-16 14:21:27,798 [myid:] - INFO [main:ZooKeeper@442] - > Initiating > client connection, connectString=zookeeper:2281 sessionTimeout=3 > watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@531d72ca > Exception in thread "main" java.io.IOException: Couldn't instantiate > org.apache.zookeeper.ClientCnxnSocketNetty > at > org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1851) > at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:453) > at > org.apache.zookeeper.ZooKeeperMain.connectToZK(ZooKeeperMain.java:283) > at org.apache.zookeeper.ZooKeeperMain.(ZooKeeperMain.java:297) > at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:290) > Caused by: java.lang.ClassNotFoundException: > org.apache.zookeeper.ClientCnxnSocketNetty > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1848) > ... 4 more > > 8.Zookeeper is working fine on 2181 > 9.I tried to connect Kafka to Zookeeper on port 2281. Getting below > error > > [2020-01-16 09:12:07,477] INFO Initiating client connection, > connectString=zookeeper:2281 sessionTimeout=6000 > > watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@5c33f1a9 > (org.apache.zookeeper.ZooKeeper) > [2020-01-16 09:12:07,488] INFO [ZooKeeperClient] Waiting until > connected. > (kafka.zookeeper.ZooKeeperClient) > [2020-01-16 09:12:07,489] INFO Opening socket connection to server > zookeeper/172.16.13.2:2281. Will not attempt to authenticate using > SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > [2020-01-16 09:12:07,493] INFO Socket error occurred: zookeeper/ > 172.16.13.2:2281: Connection refused (org.apache.zookeeper.ClientCnxn) > [2020-01-16 09:12:08,599] INFO Opening socket connection to server > zookeeper/172.16.13.2:2281. Will not attempt to authenticate using > SASL > (unknown error) (org.apache.zookeeper.ClientCnxn) > > Please help and advice. > > Regards, > Praveen Kumar K S > +91-9986855625 > > > > > >
Re: ZooKeeper in secure mode
Praveen In order to use Netty it is better for you to use 3.5.6 that contains Netty 4, ZooKeeper 3.4.x uses the deprecated Netty 3. For TSL, and it is known to have security flaws and it is no more maintained Btw your problem looks like there is a missing class and it is weird Enrico Il giorno 16/01/20, 10:25 "Praveen Kumar K S" ha scritto: Hello, I'm looking for help on enabling authentication in zookeeper. Please note below approach I have tried. 1. I followed https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide 2. I'm deploying zookeeper as single node using docker 3. Zookeeper version is 3.4.13 4. Below are some important environmental variables in zookeeper container CLIENT_JVMFLAGS=-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty -Dzookeeper.client.secure=true -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks -Dzookeeper.ssl.keyStore.password=XX@123 -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks -Dzookeeper.ssl.trustStore.password=XX@123 SERVER_JVMFLAGS=-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks -Dzookeeper.ssl.keyStore.password=XX@123 -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks -Dzookeeper.ssl.trustStore.password=XX@123 zookeeper.serverCnxnFactory="org.apache.zookeeper.server.NettyServerCnxnFactory" 5. Below is conf file server.1=0.0.0.0:2888:3888 secureClientPort=2281 initLimit=5 syncLimit=2 tickTime=2000 clientPort=2181 clientPortAddress=zookeeper dataLogDir=/opt/vault/zookeeper/logs dataDir=/opt/vault/zookeeper/data 6. Zookeeper is healthy 7. I tried connecting to Zookeeper server from my machine using zkCli.sh. But getting below error 2020-01-16 14:21:27,798 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=zookeeper:2281 sessionTimeout=3 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@531d72ca Exception in thread "main" java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNetty at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1851) at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:453) at org.apache.zookeeper.ZooKeeperMain.connectToZK(ZooKeeperMain.java:283) at org.apache.zookeeper.ZooKeeperMain.(ZooKeeperMain.java:297) at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:290) Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ClientCnxnSocketNetty at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1848) ... 4 more 8.Zookeeper is working fine on 2181 9.I tried to connect Kafka to Zookeeper on port 2281. Getting below error [2020-01-16 09:12:07,477] INFO Initiating client connection, connectString=zookeeper:2281 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@5c33f1a9 (org.apache.zookeeper.ZooKeeper) [2020-01-16 09:12:07,488] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient) [2020-01-16 09:12:07,489] INFO Opening socket connection to server zookeeper/172.16.13.2:2281. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2020-01-16 09:12:07,493] INFO Socket error occurred: zookeeper/ 172.16.13.2:2281: Connection refused (org.apache.zookeeper.ClientCnxn) [2020-01-16 09:12:08,599] INFO Opening socket connection to server zookeeper/172.16.13.2:2281. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) Please help and advice. Regards, Praveen Kumar K S +91-9986855625 CONFIDENTIALITY & PRIVACY NOTICE This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties. The use of this e-mail is only for professional purposes; there is no guarantee that
ZooKeeper in secure mode
Hello, I'm looking for help on enabling authentication in zookeeper. Please note below approach I have tried. 1. I followed https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide 2. I'm deploying zookeeper as single node using docker 3. Zookeeper version is 3.4.13 4. Below are some important environmental variables in zookeeper container CLIENT_JVMFLAGS=-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty -Dzookeeper.client.secure=true -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks -Dzookeeper.ssl.keyStore.password=XX@123 -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks -Dzookeeper.ssl.trustStore.password=XX@123 SERVER_JVMFLAGS=-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory -Dzookeeper.ssl.keyStore.location=/opt/vault/zookeeper/ssl/KeyStore.jks -Dzookeeper.ssl.keyStore.password=XX@123 -Dzookeeper.ssl.trustStore.location=/opt/vault/zookeeper/ssl/truststore.jks -Dzookeeper.ssl.trustStore.password=XX@123 zookeeper.serverCnxnFactory="org.apache.zookeeper.server.NettyServerCnxnFactory" 5. Below is conf file server.1=0.0.0.0:2888:3888 secureClientPort=2281 initLimit=5 syncLimit=2 tickTime=2000 clientPort=2181 clientPortAddress=zookeeper dataLogDir=/opt/vault/zookeeper/logs dataDir=/opt/vault/zookeeper/data 6. Zookeeper is healthy 7. I tried connecting to Zookeeper server from my machine using zkCli.sh. But getting below error 2020-01-16 14:21:27,798 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=zookeeper:2281 sessionTimeout=3 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@531d72ca Exception in thread "main" java.io.IOException: Couldn't instantiate org.apache.zookeeper.ClientCnxnSocketNetty at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1851) at org.apache.zookeeper.ZooKeeper.(ZooKeeper.java:453) at org.apache.zookeeper.ZooKeeperMain.connectToZK(ZooKeeperMain.java:283) at org.apache.zookeeper.ZooKeeperMain.(ZooKeeperMain.java:297) at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:290) Caused by: java.lang.ClassNotFoundException: org.apache.zookeeper.ClientCnxnSocketNetty at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.zookeeper.ZooKeeper.getClientCnxnSocket(ZooKeeper.java:1848) ... 4 more 8.Zookeeper is working fine on 2181 9.I tried to connect Kafka to Zookeeper on port 2281. Getting below error [2020-01-16 09:12:07,477] INFO Initiating client connection, connectString=zookeeper:2281 sessionTimeout=6000 watcher=kafka.zookeeper.ZooKeeperClient$ZooKeeperClientWatcher$@5c33f1a9 (org.apache.zookeeper.ZooKeeper) [2020-01-16 09:12:07,488] INFO [ZooKeeperClient] Waiting until connected. (kafka.zookeeper.ZooKeeperClient) [2020-01-16 09:12:07,489] INFO Opening socket connection to server zookeeper/172.16.13.2:2281. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2020-01-16 09:12:07,493] INFO Socket error occurred: zookeeper/ 172.16.13.2:2281: Connection refused (org.apache.zookeeper.ClientCnxn) [2020-01-16 09:12:08,599] INFO Opening socket connection to server zookeeper/172.16.13.2:2281. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) Please help and advice. Regards, Praveen Kumar K S +91-9986855625