Hi Mate, Thanks for suggesting these options in detail
1) We are already using AdminServer as an alternate to the 4LW, hopefully we'll look at modifying the downstream systems to use REST instead of the 4LW commands. 2) Added "clientPort=2181" back to the configs and tested "srvr" & other whitelisted 4LW commands and they all work now :) 3) When I configure the same port "2281" for both secure and unsecure communication with "client.portUnification=true", the JVM exits with Bind exception stating the "Address already in use" & unable to start ZK. For short term, I think we'd run a mixed-mode communication like you mentioned in option 2 & whitelist only specific 4LW commands required and not all. Appreciate if someone can confirm if the 4LW is expected to work against secure client port or not so we can update the doc accordingly. Thanks again! Regards, Karthick On Mon, 30 Mar 2020 at 09:30, Szalay-Bekő Máté <szalay.beko.m...@gmail.com> wrote: > Hi Karthick, > > I am not sure if "echo srvr | nc localhost 2281" is expected to work > against the secure client port. I don't think so, but maybe others know > better. I think you have the following options: > > 1) use the admin server which is a HTTP interface where the 4LW commands > are available on a REST protocol (see > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_adminserver > ) > > 2) if AdminServer is not an option for you, then you can configure > ZooKeeper to use both secure and unsecure ports. And use the unsecure port > for 4LW commands, while use the secure port for the rest of the traffic. > E.g.: > clientPort=2281 > secureClientPort=2282 > > 3) you can even configure ZooKeeper to use the same port for both TLS and > unsecure communication. I haven't used 4LW commands with port unification, > but I assume it works: > client.portUnification=true > > I hope some of these options will work for you. > > Kind regards, > Mate > > On Mon, Mar 30, 2020 at 12:24 AM karthick rn <karthick.narend...@gmail.com > > > wrote: > > > Hello, > > > > After configuring TLS, running "echo srvr | nc localhost 2281" or any > other > > 4LW doesn’t show any output. The below messages are printed on the ZK log > > whilst running the ‘srvr’ command. Also tried adding > > "4lw.commands.whitelist=*" to zoo.cfg but still no difference. However, > > disabling TLS I'm able to see all 4LW working as expected. > > > > Let me know if this is a known issue when TLS is enabled? I'm using ZK > v3.6 > > and have seen the same behaviour with v3.5.6 & 3.5.7. > > > > I have shared my Quorum TLS configs at the bottom, in-case if you want to > > check if I'm missing something. Many thanks > > > > > > zookeeper.log: > > > > > > 2020-03-29 21:09:27,079 [myid:1] - ERROR > > [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CertificateVerifier@434] - > > Unsuccessful handshake with session 0x0 > > > > 2020-03-29 21:09:27,083 [myid:1] - WARN > > [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CnxnChannelHandler@273] - > > Exception caught > > > > io.netty.handler.codec.DecoderException: > > io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: > > 737276720a > > > > at > > > > > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) > > > > at > > > > > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) > > > > at > > > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) > > > > at > > > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) > > > > at > > > > > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) > > > > at > > > > > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) > > > > at > > > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) > > > > at > > > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) > > > > at > > > > > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) > > > > at > > > > > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) > > > > at > > > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) > > > > at > > > > > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) > > > > at > > > > > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) > > > > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) > > > > at > > > > > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > > > > at > > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > > > > at > > > > > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > > > > at java.base/java.lang.Thread.run(Thread.java:834) > > > > Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS > > record: 737276720a > > > > at > > io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1198) > > > > at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266) > > > > at > > > > > io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498) > > > > at > > > > > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437) > > > > ... 17 more > > > > > > > > conf/zoo.cfg: > > > > > > > > # Server configuration > > > > secureClientPort=2281 > > > > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > > > > > # Quorum configuration > > > > sslQuorum=true > > > > ssl.quorum.keyStore.location=</path/to/keystore.jks> > > > > ssl.quorum.keyStore.password=<password> > > > > ssl.quorum.trustStore.location=</path/to/truststore.jks> > > > > ssl.quorum.trustStore.password=<password> > > > > > > > > bin/zkEnv.sh > > > > > > > > > > > SERVER_JVMFLAGS="-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > \ > > > > -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \ > > > > -Dzookeeper.ssl.keyStore.password=<password>\ > > > > -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \ > > > > -Dzookeeper.ssl.trustStore.password=<password>" > > > > > > > > > > > CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > \ > > > > -Dzookeeper.client.secure=true \ > > > > -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \ > > > > -Dzookeeper.ssl.keyStore.password=<password>\ > > > > -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \ > > > > -Dzookeeper.ssl.trustStore.password=<password>" > > > > > > > > - Karthick > > >
