Hi Karthick, The following command works for me on the secure port (1181):
(echo "srvr"; sleep 1) | openssl s_client -connect zkhost:1181 -cert cert.pem -key ./key.pem I had to add sleep, because openssl client closes the connection as soon as stdin ends. There’s no point trying non-secure communication on the secure port as it’s currently not unified. Andor > On 2020. Mar 31., at 15:22, karthick rn <karthick.narend...@gmail.com> wrote: > > Thanks Enrico for sharing the jira. This is great! > > With the below config, I'm now able to run the 4LW commands successfully, > also the downstream systems that was relying on the 4LW commands started > displaying the metrics. Thanks for your help. > > #secureClientPort=2281 > > clientPort=2281 > > client.portUnification=True > > > - Karthick > > > > On Mon, 30 Mar 2020 at 21:59, Enrico Olivelli <eolive...@gmail.com> wrote: > >> You may be interested in Port unification, contributed by Facebook: >> >> https://issues.apache.org/jira/browse/ZOOKEEPER-3388 >> https://issues.apache.org/jira/browse/ZOOKEEPER-3371 >> >> Enrico >> >> Il giorno lun 30 mar 2020 alle ore 13:33 karthick rn >> <karthick.narend...@gmail.com> ha scritto: >>> >>> Hi Mate, >>> >>> Thanks for suggesting these options in detail >>> >>> 1) We are already using AdminServer as an alternate to the 4LW, hopefully >>> we'll look at modifying the downstream systems to use REST instead of the >>> 4LW commands. >>> >>> 2) Added "clientPort=2181" back to the configs and tested "srvr" & other >>> whitelisted 4LW commands and they all work now :) >>> >>> 3) When I configure the same port "2281" for both secure and unsecure >>> communication with "client.portUnification=true", the JVM exits with Bind >>> exception stating the "Address already in use" & unable to start ZK. >>> >>> For short term, I think we'd run a mixed-mode communication like you >>> mentioned in option 2 & whitelist only specific 4LW commands required and >>> not all. >>> >>> Appreciate if someone can confirm if the 4LW is expected to work against >>> secure client port or not so we can update the doc accordingly. Thanks >>> again! >>> >>> Regards, >>> Karthick >>> >>> On Mon, 30 Mar 2020 at 09:30, Szalay-Bekő Máté < >> szalay.beko.m...@gmail.com> >>> wrote: >>> >>>> Hi Karthick, >>>> >>>> I am not sure if "echo srvr | nc localhost 2281" is expected to work >>>> against the secure client port. I don't think so, but maybe others know >>>> better. I think you have the following options: >>>> >>>> 1) use the admin server which is a HTTP interface where the 4LW >> commands >>>> are available on a REST protocol (see >>>> >> https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_adminserver >>>> ) >>>> >>>> 2) if AdminServer is not an option for you, then you can configure >>>> ZooKeeper to use both secure and unsecure ports. And use the unsecure >> port >>>> for 4LW commands, while use the secure port for the rest of the >> traffic. >>>> E.g.: >>>> clientPort=2281 >>>> secureClientPort=2282 >>>> >>>> 3) you can even configure ZooKeeper to use the same port for both TLS >> and >>>> unsecure communication. I haven't used 4LW commands with port >> unification, >>>> but I assume it works: >>>> client.portUnification=true >>>> >>>> I hope some of these options will work for you. >>>> >>>> Kind regards, >>>> Mate >>>> >>>> On Mon, Mar 30, 2020 at 12:24 AM karthick rn < >> karthick.narend...@gmail.com >>>>> >>>> wrote: >>>> >>>>> Hello, >>>>> >>>>> After configuring TLS, running "echo srvr | nc localhost 2281" or any >>>> other >>>>> 4LW doesn’t show any output. The below messages are printed on the >> ZK log >>>>> whilst running the ‘srvr’ command. Also tried adding >>>>> "4lw.commands.whitelist=*" to zoo.cfg but still no difference. >> However, >>>>> disabling TLS I'm able to see all 4LW working as expected. >>>>> >>>>> Let me know if this is a known issue when TLS is enabled? I'm using >> ZK >>>> v3.6 >>>>> and have seen the same behaviour with v3.5.6 & 3.5.7. >>>>> >>>>> I have shared my Quorum TLS configs at the bottom, in-case if you >> want to >>>>> check if I'm missing something. Many thanks >>>>> >>>>> >>>>> zookeeper.log: >>>>> >>>>> >>>>> 2020-03-29 21:09:27,079 [myid:1] - ERROR >>>>> [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CertificateVerifier@434] >> - >>>>> Unsuccessful handshake with session 0x0 >>>>> >>>>> 2020-03-29 21:09:27,083 [myid:1] - WARN >>>>> [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CnxnChannelHandler@273] >> - >>>>> Exception caught >>>>> >>>>> io.netty.handler.codec.DecoderException: >>>>> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: >>>>> 737276720a >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) >>>>> >>>>> at >>>>> >>>> >> io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) >>>>> >>>>> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) >>>>> >>>>> at >>>>> >> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) >>>>> >>>>> at java.base/java.lang.Thread.run(Thread.java:834) >>>>> >>>>> Caused by: io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS >>>>> record: 737276720a >>>>> >>>>> at >>>>> >> io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1198) >>>>> >>>>> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498) >>>>> >>>>> at >>>>> >>>>> >>>> >> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437) >>>>> >>>>> ... 17 more >>>>> >>>>> >>>>> >>>>> conf/zoo.cfg: >>>>> >>>>> >>>>> >>>>> # Server configuration >>>>> >>>>> secureClientPort=2281 >>>>> >>>>> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >>>>> >>>>> >>>>> >>>>> # Quorum configuration >>>>> >>>>> sslQuorum=true >>>>> >>>>> ssl.quorum.keyStore.location=</path/to/keystore.jks> >>>>> >>>>> ssl.quorum.keyStore.password=<password> >>>>> >>>>> ssl.quorum.trustStore.location=</path/to/truststore.jks> >>>>> >>>>> ssl.quorum.trustStore.password=<password> >>>>> >>>>> >>>>> >>>>> bin/zkEnv.sh >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >> SERVER_JVMFLAGS="-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >>>>> \ >>>>> >>>>> -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \ >>>>> >>>>> -Dzookeeper.ssl.keyStore.password=<password>\ >>>>> >>>>> -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \ >>>>> >>>>> -Dzookeeper.ssl.trustStore.password=<password>" >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >> CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >>>>> \ >>>>> >>>>> -Dzookeeper.client.secure=true \ >>>>> >>>>> -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \ >>>>> >>>>> -Dzookeeper.ssl.keyStore.password=<password>\ >>>>> >>>>> -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \ >>>>> >>>>> -Dzookeeper.ssl.trustStore.password=<password>" >>>>> >>>>> >>>>> >>>>> - Karthick >>>>> >>>> >>
