Re: ZK client session expired after implementing SSL
When I turned on Debug logs I can see the requests are getting throttled and even PING request from client is throttled. Later event buffer was full and it discarded the Autoread.enable message. Is there a way to avoid discarding of message or increase the throttling limit ? The cluster memory is well under controlled. Thanks Srikant Kalani On Mon, 13 Jul 2020 at 11:21 PM, shrikant kalani wrote: > Adding one more email list > > On Mon, 13 Jul 2020 at 10:49 PM, shrikant kalani > wrote: > >> Hi >> >> We are seeing a very uncommon behaviour. We implemented SSL for quorum >> communication in version 3.5.5. After the change we are seeing ZK client >> applications are frequently getting crashed with session expired message. >> While there are no signs of GC in the application, it is hard to figure out >> why ZK server is disconnecting those application. >> >> Any thoughts ? >> >> Thanks >> Srikant Kalani >> >
Re: ZK client session expired after implementing SSL
Adding one more email list On Mon, 13 Jul 2020 at 10:49 PM, shrikant kalani wrote: > Hi > > We are seeing a very uncommon behaviour. We implemented SSL for quorum > communication in version 3.5.5. After the change we are seeing ZK client > applications are frequently getting crashed with session expired message. > While there are no signs of GC in the application, it is hard to figure out > why ZK server is disconnecting those application. > > Any thoughts ? > > Thanks > Srikant Kalani >
Re: Zookeeper session expiration
I am facing a similar issue in my application. Zookeeper Server Version 3.5.5 I implemented SSL ( server to server ) in quorum communication. After that ZK client frequently receives session timeouts. When I turned off SSL then application is behaving normally and there are no timeouts. Any thoughts ? Thanks Srikant Kalani -- Sent from: http://zookeeper-user.578899.n2.nabble.com/
ZK client session expired after implementing SSL
Hi We are seeing a very uncommon behaviour. We implemented SSL for quorum communication in version 3.5.5. After the change we are seeing ZK client applications are frequently getting crashed with session expired message. While there are no signs of GC in the application, it is hard to figure out why ZK server is disconnecting those application. Any thoughts ? Thanks Srikant Kalani
Re: upgrade from 3.4.5 to 3.5.6
It looks like we ported it to 3.5. See the subtask https://issues.apache.org/jira/browse/ZOOKEEPER-2792 Enrico Il giorno 13/07/20, 10:37 "kuldeep singh" ha scritto: Hi Team, I appreciate it if I will get a response as soon as possible, as I am stuck at this point. Thanks, - Kuldeep Singh Budania On Mon, Jul 13, 2020 at 11:10 AM kuldeep singh wrote: > Hi Team, > > Server to Server communication is not supported in 3.5.6 version as per > below JIRA issue? > > https://issues.apache.org/jira/browse/ZOOKEEPER-2639 > > Thanks, > - > Kuldeep Singh Budania > > > > On Thu, Jul 2, 2020 at 4:24 PM kuldeep singh > wrote: > >> Thanks for the reply. >> >> Now my ZKCli cmd is working fine as we use some our customized >> authentication and we resolve the issue. >> >> Now I am going to implement Server to Server communication. >> >> Thanks, >> - >> Kuldeep Singh Budania >> >> >> >> On Thu, Jul 2, 2020 at 3:53 PM Szalay-Bekő Máté < >> szalay.beko.m...@gmail.com> wrote: >> >>> I think SSL is working for you already... If you managed to start the >>> zkCli.sh and be able to connect to ZooKeeper on the secure port and issue >>> any kind of command (like: " ls / "), then the wire encryption is working >>> and your server/client communication is secured by ZooKeeper. >>> >>> Why you want to run the following command? >>> addauth ztpasswd zooadmin: >>> >>> Do you also want to configure a superDigest user in ZooKeeper? Please >>> note >>> that this command is independent from SSL. If you need to create a >>> username-password pair for digest authentication then please use the >>> command in the following way: >>> addauth digest zooadmin:yourSuperSecretPassword >>> >>> Kind regards, >>> Mate >>> >>> On Thu, Jul 2, 2020 at 6:59 AM kuldeep singh >>> wrote: >>> >>> > 1. sh zkCli.sh --config /etc/zookeeper -server localhost:2281 >>> > >>> > 2. addauth ztpasswd zooadmin: >>> > >>> > >>> > Thanks, >>> > - >>> > Kuldeep Singh Budania >>> > >>> > >>> > >>> > On Thu, Jul 2, 2020 at 9:56 AM kuldeep singh < >>> kuldeep.sing...@gmail.com> >>> > wrote: >>> > >>> > > Hi Team, >>> > > >>> > > Any update on this? >>> > > >>> > > Thanks, >>> > > - >>> > > Kuldeep Singh Budania >>> > > >>> > > >>> > > >>> > > On Wed, Jul 1, 2020 at 6:43 PM kuldeep singh < >>> kuldeep.sing...@gmail.com> >>> > > wrote: >>> > > >>> > >> Sorry this is my bad, there were server setting like below >>> > >> >>> > >> export SERVER_JVMFLAGS=" >>> > >> >>> > >> >>> > >> >>> > >>> -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >>> > >> >>> > >> >>> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >>> > >> -Dzookeeper.ssl.keyStore.password=testpass >>> > >> >>> > >>> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >>> > >> -Dzookeeper.ssl.trustStore.password=testpass" >>> > >> >>> > >> >>> > >> >>> > >> export CLIENT_JVMFLAGS=" >>> > >> >>> > >> >>> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >>> > >> -Dzookeeper.client.secure=true >>> > >> >>> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >>> > >> -Dzookeeper.ssl.keyStore.password=testpass >>> > >> >>> > >>> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >>> > >> -Dzookeeper.ssl.trustStore.password=testpass" >>> > >> >>> > >> I want to have SSL between client to server communication >>> > >> >>> > >> I am already following the same link which you have shared with me >>> but >>> > >> that is not working. >>> > >> >>> > >> Zoo.cfg >>> > >> >>> > >> secureClientPort=2281 >>> > >> initLimit=10 >>> > >> syncLimit=5 >>> > >> dataDir=/var/lib/zookeeper/data >>> > >> tickTime=2000 >>> > >> autopurge.snapRetainCount=3 >>> > >> autopurge.purgeInterval=1 >>> > >> admin.enableServer=false >>> > >> standaloneEnabled=false >>> > >> jute.maxbuffer=2147483648 >>> > >> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >>> > >> server.1=host1_priv:10288:10388 >>> > >> server.2=host2_priv:10288:10388 >>> > >> server.3=host3_priv:10288:10388 >>> > >> >>> > >> >>> > >> command to connect using zkcli >>> > >> >>> > >> 1. zkcli zoo.cfg localhost:2281 >>> > >> 2. addauth ztpasswd usernaem:password >>> > >> >>> > >> after second step we are getting
Re: upgrade from 3.4.5 to 3.5.6
Hi Team, I appreciate it if I will get a response as soon as possible, as I am stuck at this point. Thanks, - Kuldeep Singh Budania On Mon, Jul 13, 2020 at 11:10 AM kuldeep singh wrote: > Hi Team, > > Server to Server communication is not supported in 3.5.6 version as per > below JIRA issue? > > https://issues.apache.org/jira/browse/ZOOKEEPER-2639 > > Thanks, > - > Kuldeep Singh Budania > > > > On Thu, Jul 2, 2020 at 4:24 PM kuldeep singh > wrote: > >> Thanks for the reply. >> >> Now my ZKCli cmd is working fine as we use some our customized >> authentication and we resolve the issue. >> >> Now I am going to implement Server to Server communication. >> >> Thanks, >> - >> Kuldeep Singh Budania >> >> >> >> On Thu, Jul 2, 2020 at 3:53 PM Szalay-Bekő Máté < >> szalay.beko.m...@gmail.com> wrote: >> >>> I think SSL is working for you already... If you managed to start the >>> zkCli.sh and be able to connect to ZooKeeper on the secure port and issue >>> any kind of command (like: " ls / "), then the wire encryption is working >>> and your server/client communication is secured by ZooKeeper. >>> >>> Why you want to run the following command? >>> addauth ztpasswd zooadmin: >>> >>> Do you also want to configure a superDigest user in ZooKeeper? Please >>> note >>> that this command is independent from SSL. If you need to create a >>> username-password pair for digest authentication then please use the >>> command in the following way: >>> addauth digest zooadmin:yourSuperSecretPassword >>> >>> Kind regards, >>> Mate >>> >>> On Thu, Jul 2, 2020 at 6:59 AM kuldeep singh >>> wrote: >>> >>> > 1. sh zkCli.sh --config /etc/zookeeper -server localhost:2281 >>> > >>> > 2. addauth ztpasswd zooadmin: >>> > >>> > >>> > Thanks, >>> > - >>> > Kuldeep Singh Budania >>> > >>> > >>> > >>> > On Thu, Jul 2, 2020 at 9:56 AM kuldeep singh < >>> kuldeep.sing...@gmail.com> >>> > wrote: >>> > >>> > > Hi Team, >>> > > >>> > > Any update on this? >>> > > >>> > > Thanks, >>> > > - >>> > > Kuldeep Singh Budania >>> > > >>> > > >>> > > >>> > > On Wed, Jul 1, 2020 at 6:43 PM kuldeep singh < >>> kuldeep.sing...@gmail.com> >>> > > wrote: >>> > > >>> > >> Sorry this is my bad, there were server setting like below >>> > >> >>> > >> export SERVER_JVMFLAGS=" >>> > >> >>> > >> >>> > >> >>> > >>> -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >>> > >> >>> > >> >>> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >>> > >> -Dzookeeper.ssl.keyStore.password=testpass >>> > >> >>> > >>> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >>> > >> -Dzookeeper.ssl.trustStore.password=testpass" >>> > >> >>> > >> >>> > >> >>> > >> export CLIENT_JVMFLAGS=" >>> > >> >>> > >> >>> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >>> > >> -Dzookeeper.client.secure=true >>> > >> >>> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >>> > >> -Dzookeeper.ssl.keyStore.password=testpass >>> > >> >>> > >>> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >>> > >> -Dzookeeper.ssl.trustStore.password=testpass" >>> > >> >>> > >> I want to have SSL between client to server communication >>> > >> >>> > >> I am already following the same link which you have shared with me >>> but >>> > >> that is not working. >>> > >> >>> > >> Zoo.cfg >>> > >> >>> > >> secureClientPort=2281 >>> > >> initLimit=10 >>> > >> syncLimit=5 >>> > >> dataDir=/var/lib/zookeeper/data >>> > >> tickTime=2000 >>> > >> autopurge.snapRetainCount=3 >>> > >> autopurge.purgeInterval=1 >>> > >> admin.enableServer=false >>> > >> standaloneEnabled=false >>> > >> jute.maxbuffer=2147483648 >>> > >> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >>> > >> server.1=host1_priv:10288:10388 >>> > >> server.2=host2_priv:10288:10388 >>> > >> server.3=host3_priv:10288:10388 >>> > >> >>> > >> >>> > >> command to connect using zkcli >>> > >> >>> > >> 1. zkcli zoo.cfg localhost:2281 >>> > >> 2. addauth ztpasswd usernaem:password >>> > >> >>> > >> after second step we are getting below error >>> > >> >>> > >> WatchedEvent state:AuthFailed type:None path:null >>> > >> >>> > >> >>> > >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN >>> > >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication >>> > provider >>> > >> for scheme: ztpasswd has x509 ip digest* >>> > >> >>> > >> Thanks, >>> > >> - >>> > >> Kuldeep Singh Budania >>> > >> >>> > >> >>> > >> >>> > >> On Wed, Jul 1, 2020 at 6:25 PM Szalay-Bekő Máté < >>> > >> szalay.beko.m...@gmail.com> wrote: >>> > >> >>> > >>> > No authentication provider for scheme: ztpasswd has x509 ip >>> digest* >>> > >>> >>> > >>> This suggest you have some configuration error... Where did you >>> use the >>> > >>> "ztpasswd" string in your configs / commands? >>> > >>> >>> > >>> On Wed,
