Hi Andor, I've tried the openssl command you shared but unable to get it working, may be something to do with converting to PEM format the keys and certs. I'll look into this.
> There’s no point trying non-secure communication on the secure port as it’s currently not unified. I'm not getting, please can you explain it? Thanks, Karthick On Tue, 31 Mar 2020 at 15:50, Patrick Hunt <ph...@apache.org> wrote: > We (Karthick can you? :-) ) should add this to the docs. PR would be great! > https://cwiki.apache.org/confluence/display/ZOOKEEPER/HowToContribute > > Thanks, > > Patrick > > On Tue, Mar 31, 2020 at 7:17 AM Andor Molnar <an...@apache.org> wrote: > > > Hi Karthick, > > > > The following command works for me on the secure port (1181): > > > > (echo "srvr"; sleep 1) | openssl s_client -connect zkhost:1181 -cert > > cert.pem -key ./key.pem > > > > I had to add sleep, because openssl client closes the connection as soon > > as stdin ends. > > There’s no point trying non-secure communication on the secure port as > > it’s currently not unified. > > > > Andor > > > > > > > > > > > > > On 2020. Mar 31., at 15:22, karthick rn <karthick.narend...@gmail.com> > > wrote: > > > > > > Thanks Enrico for sharing the jira. This is great! > > > > > > With the below config, I'm now able to run the 4LW commands > successfully, > > > also the downstream systems that was relying on the 4LW commands > started > > > displaying the metrics. Thanks for your help. > > > > > > #secureClientPort=2281 > > > > > > clientPort=2281 > > > > > > client.portUnification=True > > > > > > > > > - Karthick > > > > > > > > > > > > On Mon, 30 Mar 2020 at 21:59, Enrico Olivelli <eolive...@gmail.com> > > wrote: > > > > > >> You may be interested in Port unification, contributed by Facebook: > > >> > > >> https://issues.apache.org/jira/browse/ZOOKEEPER-3388 > > >> https://issues.apache.org/jira/browse/ZOOKEEPER-3371 > > >> > > >> Enrico > > >> > > >> Il giorno lun 30 mar 2020 alle ore 13:33 karthick rn > > >> <karthick.narend...@gmail.com> ha scritto: > > >>> > > >>> Hi Mate, > > >>> > > >>> Thanks for suggesting these options in detail > > >>> > > >>> 1) We are already using AdminServer as an alternate to the 4LW, > > hopefully > > >>> we'll look at modifying the downstream systems to use REST instead of > > the > > >>> 4LW commands. > > >>> > > >>> 2) Added "clientPort=2181" back to the configs and tested "srvr" & > > other > > >>> whitelisted 4LW commands and they all work now :) > > >>> > > >>> 3) When I configure the same port "2281" for both secure and unsecure > > >>> communication with "client.portUnification=true", the JVM exits with > > Bind > > >>> exception stating the "Address already in use" & unable to start ZK. > > >>> > > >>> For short term, I think we'd run a mixed-mode communication like you > > >>> mentioned in option 2 & whitelist only specific 4LW commands required > > and > > >>> not all. > > >>> > > >>> Appreciate if someone can confirm if the 4LW is expected to work > > against > > >>> secure client port or not so we can update the doc accordingly. > Thanks > > >>> again! > > >>> > > >>> Regards, > > >>> Karthick > > >>> > > >>> On Mon, 30 Mar 2020 at 09:30, Szalay-Bekő Máté < > > >> szalay.beko.m...@gmail.com> > > >>> wrote: > > >>> > > >>>> Hi Karthick, > > >>>> > > >>>> I am not sure if "echo srvr | nc localhost 2281" is expected to work > > >>>> against the secure client port. I don't think so, but maybe others > > know > > >>>> better. I think you have the following options: > > >>>> > > >>>> 1) use the admin server which is a HTTP interface where the 4LW > > >> commands > > >>>> are available on a REST protocol (see > > >>>> > > >> > > > https://zookeeper.apache.org/doc/r3.6.0/zookeeperAdmin.html#sc_adminserver > > >>>> ) > > >>>> > > >>>> 2) if AdminServer is not an option for you, then you can configure > > >>>> ZooKeeper to use both secure and unsecure ports. And use the > unsecure > > >> port > > >>>> for 4LW commands, while use the secure port for the rest of the > > >> traffic. > > >>>> E.g.: > > >>>> clientPort=2281 > > >>>> secureClientPort=2282 > > >>>> > > >>>> 3) you can even configure ZooKeeper to use the same port for both > TLS > > >> and > > >>>> unsecure communication. I haven't used 4LW commands with port > > >> unification, > > >>>> but I assume it works: > > >>>> client.portUnification=true > > >>>> > > >>>> I hope some of these options will work for you. > > >>>> > > >>>> Kind regards, > > >>>> Mate > > >>>> > > >>>> On Mon, Mar 30, 2020 at 12:24 AM karthick rn < > > >> karthick.narend...@gmail.com > > >>>>> > > >>>> wrote: > > >>>> > > >>>>> Hello, > > >>>>> > > >>>>> After configuring TLS, running "echo srvr | nc localhost 2281" or > any > > >>>> other > > >>>>> 4LW doesn’t show any output. The below messages are printed on the > > >> ZK log > > >>>>> whilst running the ‘srvr’ command. Also tried adding > > >>>>> "4lw.commands.whitelist=*" to zoo.cfg but still no difference. > > >> However, > > >>>>> disabling TLS I'm able to see all 4LW working as expected. > > >>>>> > > >>>>> Let me know if this is a known issue when TLS is enabled? I'm using > > >> ZK > > >>>> v3.6 > > >>>>> and have seen the same behaviour with v3.5.6 & 3.5.7. > > >>>>> > > >>>>> I have shared my Quorum TLS configs at the bottom, in-case if you > > >> want to > > >>>>> check if I'm missing something. Many thanks > > >>>>> > > >>>>> > > >>>>> zookeeper.log: > > >>>>> > > >>>>> > > >>>>> 2020-03-29 21:09:27,079 [myid:1] - ERROR > > >>>>> > [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CertificateVerifier@434 > > ] > > >> - > > >>>>> Unsuccessful handshake with session 0x0 > > >>>>> > > >>>>> 2020-03-29 21:09:27,083 [myid:1] - WARN > > >>>>> > [nioEventLoopGroup-4-1:NettyServerCnxnFactory$CnxnChannelHandler@273 > > ] > > >> - > > >>>>> Exception caught > > >>>>> > > >>>>> io.netty.handler.codec.DecoderException: > > >>>>> io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: > > >>>>> 737276720a > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:377) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) > > >>>>> > > >>>>> at > > >>>>> > > >>>> > > >> > > > io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) > > >>>>> > > >>>>> at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) > > >>>>> > > >>>>> at > > >>>>> > > >> > > io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > > >>>>> > > >>>>> at java.base/java.lang.Thread.run(Thread.java:834) > > >>>>> > > >>>>> Caused by: io.netty.handler.ssl.NotSslRecordException: not an > SSL/TLS > > >>>>> record: 737276720a > > >>>>> > > >>>>> at > > >>>>> > > >> > > io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1198) > > >>>>> > > >>>>> at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1266) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:498) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >> > > > io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437) > > >>>>> > > >>>>> ... 17 more > > >>>>> > > >>>>> > > >>>>> > > >>>>> conf/zoo.cfg: > > >>>>> > > >>>>> > > >>>>> > > >>>>> # Server configuration > > >>>>> > > >>>>> secureClientPort=2281 > > >>>>> > > >>>>> > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >>>>> > > >>>>> > > >>>>> > > >>>>> # Quorum configuration > > >>>>> > > >>>>> sslQuorum=true > > >>>>> > > >>>>> ssl.quorum.keyStore.location=</path/to/keystore.jks> > > >>>>> > > >>>>> ssl.quorum.keyStore.password=<password> > > >>>>> > > >>>>> ssl.quorum.trustStore.location=</path/to/truststore.jks> > > >>>>> > > >>>>> ssl.quorum.trustStore.password=<password> > > >>>>> > > >>>>> > > >>>>> > > >>>>> bin/zkEnv.sh > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > > SERVER_JVMFLAGS="-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >>>>> \ > > >>>>> > > >>>>> -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \ > > >>>>> > > >>>>> -Dzookeeper.ssl.keyStore.password=<password>\ > > >>>>> > > >>>>> -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \ > > >>>>> > > >>>>> -Dzookeeper.ssl.trustStore.password=<password>" > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>> > > >> > > > CLIENT_JVMFLAGS="-Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > >>>>> \ > > >>>>> > > >>>>> -Dzookeeper.client.secure=true \ > > >>>>> > > >>>>> -Dzookeeper.ssl.keyStore.location=</path/to/keystore.jks> \ > > >>>>> > > >>>>> -Dzookeeper.ssl.keyStore.password=<password>\ > > >>>>> > > >>>>> -Dzookeeper.ssl.trustStore.location=</path/to/truststore.jks> \ > > >>>>> > > >>>>> -Dzookeeper.ssl.trustStore.password=<password>" > > >>>>> > > >>>>> > > >>>>> > > >>>>> - Karthick > > >>>>> > > >>>> > > >> > > > > >
