Re: upgrade from 3.4.5 to 3.5.6

Hi, Yes, all the configurations are the same on all 5 nodes. I have checked more than 10 times. Before scaling , 3 node cluster was working fine, but scaling from 3 to 5 node, then we are facing this problem. only question is on 3 node cluster it is working fine, but after scaling it is not

Re: upgrade from 3.4.5 to 3.5.6

the config looks OK in general... - are you sure the same configs are used on all ZK servers? - does the truststores accept all keys on the keystores? (if the truststores of the old servers had to be modified, then did you restart the old servers with the updated truststores?) - did the 3 node ZK

Re: upgrade from 3.4.5 to 3.5.6

Sorry, secureClientPort=2182 Thanks, - Kuldeep Singh Budania On Wed, Oct 14, 2020 at 4:18 PM kuldeep singh wrote: > Thanks for reply > > zoo.cfg > --- > secureClientPort=2181 > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > initLimit=10 >

Re: upgrade from 3.4.5 to 3.5.6

Thanks for reply zoo.cfg --- secureClientPort=2181 serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory initLimit=10 syncLimit=5 dataDir= data directory (not mentioning exact path here) tickTime=2000 autopurge.snapRetainCount=3 autopurge.purgeInterval=1

Re: upgrade from 3.4.5 to 3.5.6

These log messages indicate that a client (or an other ZooKeeper server) is trying to connect without SSL to a ZooKeeper process that expects SSL. I assume this will be a configuration issue then. Best regards, Mate On Wed, Oct 14, 2020 at 12:30 PM kuldeep singh wrote: > Hi, > > more logs > >

Re: upgrade from 3.4.5 to 3.5.6

Hi, more logs 2020-10-14 12:25:05,106 - ERROR [nioEventLoopGroup-7-4:NettyServerCnxnFactory$CnxnChannelHandler$CertificateVerifier@257] - Unsuccessful handshake with session 0x0 2020-10-14 12:25:05,107 - WARN [nioEventLoopGroup-7-4:NettyServerCnxnFactory$CnxnChannelHandler@138] - Exception

Re: upgrade from 3.4.5 to 3.5.6

Hi, below error is coming on zookeeper logs on SSL. 2020-10-14 12:18:27,410 - WARN [nioEventLoopGroup-4-16:NettyServerCnxn@540] - Closing connection to /127.0.0.1:49470 java.io.IOException: Len error 369296128 at

Re: upgrade from 3.4.5 to 3.5.6

Sounds like a bug or a configuration issue... can you share the configs (before and after the scale-up) and the logs? also: does the truststores recognise all the keys used on all the 5 nodes? (e.g. the truststores on the old nodes accept the new keys?) Best Regards, Mate On Thu, Oct 8, 2020 at

Re: upgrade from 3.4.5 to 3.5.6

Hi, Yes, My client and server both are using certificate and have added in ZK and client as well. Thanks, - Kuldeep Singh Budania On Thu, Oct 8, 2020 at 5:56 PM Enrico Olivelli wrote: > Il giorno gio 8 ott 2020 alle ore 14:17 kuldeep singh < > kuldeep.sing...@gmail.com>

Re: upgrade from 3.4.5 to 3.5.6

Il giorno gio 8 ott 2020 alle ore 14:17 kuldeep singh < kuldeep.sing...@gmail.com> ha scritto: > Hi Team, > > I am facing one issue in SSL communication between client and zookeeper > server. > > ZK 3.5.6 version > > 1. Mi on 3 node > 2. Applying SSL and 3 nodes cluster is working fine > 3.

Re: upgrade from 3.4.5 to 3.5.6

Hi Team, I am facing one issue in SSL communication between client and zookeeper server. ZK 3.5.6 version 1. Mi on 3 node 2. Applying SSL and 3 nodes cluster is working fine 3. Scaled my cluster with 2 nodes and now my cluster have 5 nodes over SSL but after scaling my SSL is not working

Re: upgrade from 3.4.5 to 3.5.6

It looks like we ported it to 3.5. See the subtask https://issues.apache.org/jira/browse/ZOOKEEPER-2792 Enrico Il giorno 13/07/20, 10:37 "kuldeep singh" ha scritto: Hi Team, I appreciate it if I will get a response as soon as possible, as I am stuck at this point. Thanks,

Re: upgrade from 3.4.5 to 3.5.6

Hi Team, I appreciate it if I will get a response as soon as possible, as I am stuck at this point. Thanks, - Kuldeep Singh Budania On Mon, Jul 13, 2020 at 11:10 AM kuldeep singh wrote: > Hi Team, > > Server to Server communication is not supported in 3.5.6 version as

Re: upgrade from 3.4.5 to 3.5.6

Hi Team, Server to Server communication is not supported in 3.5.6 version as per below JIRA issue? https://issues.apache.org/jira/browse/ZOOKEEPER-2639 Thanks, - Kuldeep Singh Budania On Thu, Jul 2, 2020 at 4:24 PM kuldeep singh wrote: > Thanks for the reply. > > Now

Re: upgrade from 3.4.5 to 3.5.6

Thanks for the reply. Now my ZKCli cmd is working fine as we use some our customized authentication and we resolve the issue. Now I am going to implement Server to Server communication. Thanks, - Kuldeep Singh Budania On Thu, Jul 2, 2020 at 3:53 PM Szalay-Bekő Máté

Re: upgrade from 3.4.5 to 3.5.6

I think SSL is working for you already... If you managed to start the zkCli.sh and be able to connect to ZooKeeper on the secure port and issue any kind of command (like: " ls / "), then the wire encryption is working and your server/client communication is secured by ZooKeeper. Why you want to

Re: upgrade from 3.4.5 to 3.5.6

1. sh zkCli.sh --config /etc/zookeeper -server localhost:2281 2. addauth ztpasswd zooadmin: Thanks, - Kuldeep Singh Budania On Thu, Jul 2, 2020 at 9:56 AM kuldeep singh wrote: > Hi Team, > > Any update on this? > > Thanks, > - > Kuldeep Singh Budania

Re: upgrade from 3.4.5 to 3.5.6

Hi Team, Any update on this? Thanks, - Kuldeep Singh Budania On Wed, Jul 1, 2020 at 6:43 PM kuldeep singh wrote: > Sorry this is my bad, there were server setting like below > > export SERVER_JVMFLAGS=" > > >

Re: upgrade from 3.4.5 to 3.5.6

Sorry this is my bad, there were server setting like below export SERVER_JVMFLAGS=" -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks -Dzookeeper.ssl.keyStore.password=testpass

Re: upgrade from 3.4.5 to 3.5.6

> No authentication provider for scheme: ztpasswd has x509 ip digest* This suggest you have some configuration error... Where did you use the "ztpasswd" string in your configs / commands? On Wed, Jul 1, 2020 at 2:53 PM Szalay-Bekő Máté wrote: > > My ZK server is up and running in secure mode

Re: upgrade from 3.4.5 to 3.5.6

> My ZK server is up and running in secure mode What is your goal? You want to setup client-server SSL connection? see: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide (or you want to have both SSL and SASL enabled?) Anyway, please remove the following line from

Re: upgrade from 3.4.5 to 3.5.6

Hi, we have done below changes in java.env file export SERVER_JVMFLAGS=" -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks -Dzookeeper.ssl.keyStore.password=testpass

Re: upgrade from 3.4.5 to 3.5.6

Hi, My ZK server is up and running in secure mode, But When I am trying to connect to the ZK server using ZKCli, it gives the below error. WatchedEvent state:AuthFailed type:None path:null Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No

Re: upgrade from 3.4.5 to 3.5.6

I mean in zoo.cfg Not as a system property Enrico Il giorno 25/06/20, 08:19 "Enrico Olivelli - Diennea" ha scritto: Hi You have to enable Netty on the server side Something like: serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory Hope that helps

Re: upgrade from 3.4.5 to 3.5.6

Hi You have to enable Netty on the server side Something like: serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory Hope that helps Enrico Il giorno 24/06/20, 19:17 "kuldeep singh" ha scritto: Hi, I got below error while setting SSL properties in zkEnv.sh

Re: upgrade from 3.4.5 to 3.5.6

Hi, I appreciate if I will get a response as soon as possible, as I am stuck at this point. Thanks, - Kuldeep Singh Budania Software Architect On Wed, Jun 24, 2020 at 10:47 PM kuldeep singh wrote: > Hi, > > I got below error while setting SSL properties in zkEnv.sh > > >

Re: upgrade from 3.4.5 to 3.5.6

Hi, I got below error while setting SSL properties in zkEnv.sh == 2020-06-24 15:49:35,864 - INFO [main:QuorumPeerConfig@133] - Reading configuration from: /etc/zookeeper/zoo.cfg 2020-06-24 15:49:35,874 -

Re: upgrade from 3.4.5 to 3.5.6

It's the same as the normal ZooKeeper client: https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions -Jordan > On Jun 22, 2020, at 5:50 AM, kuldeep singh wrote: > > Hi Team, > How we will do

Re: upgrade from 3.4.5 to 3.5.6

Hi Team, How we will do secure communication between the Curator framework and zookeeper 3.5.6 ? I didn't get any solution right now. I appreciate it if someone could help me with the same. Thanks, - Kuldeep Singh Budania Software Architect On Fri, Apr 17, 2020 at 4:53 PM

Re: upgrade from 3.4.5 to 3.5.6

Hello Kuldeep, did you download the source from the ZooKeeper webpage, or checked out from git? Anyway, the following commands should work: wget https://downloads.apache.org/zookeeper/zookeeper-3.5.7/apache-zookeeper-3.5.7.tar.gz tar xzvf ./apache-zookeeper-3.5.7.tar.gz cd apache-zookeeper-3.5.7

Re: upgrade from 3.4.5 to 3.5.6

Hi All, I have download the zookeeper source code and run the mvn command for packaging but get problem on "zookeeper-server" directory. *command :- mvn clean package* [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project

Re: upgrade from 3.4.5 to 3.5.6

these exceptions can mean many things... I think this can be even normal duding rolling restart (as some connections get broken in this case) However, I saw cases already when exceptions like these killed receiver or sender threads in QuorumCnxManager / Leader Election in such a way that they

Re: upgrade from 3.4.5 to 3.5.6

Hi Team, I have done some POC on rolling upgrade and found below result. 1. On 1st node upgrade zookeeper . Traffic was running fine because 2 nodes are already on old zookeeper. 2. On 1st node upgrade our application and didn’t find any issue 3. On 2nd node upgrade zookeeper but

Re: upgrade from 3.4.5 to 3.5.6

+1 to what Mate said (I wrote the quoted instructions). On Tue, Mar 24, 2020 at 7:03 AM Szalay-Bekő Máté wrote: > Hi Kuldeep, > > I just want to provide you some background info about our documentation. > The reason to upgrade to 3.4.6 first is to avoid the following error: > > > 2013-01-30

Re: upgrade from 3.4.5 to 3.5.6

Hi Kuldeep, I just want to provide you some background info about our documentation. The reason to upgrade to 3.4.6 first is to avoid the following error: > 2013-01-30 11:32:10,663 [myid:2] - WARN [localhost/127.0.0.1:2784 :QuorumCnxManager@349] - Invalid server id: -65536 This error comes

Re: upgrade from 3.4.5 to 3.5.6

Hi, That guide is to upgrade to 3.5.0, which was an alpha version. A lot has changed for the first stable release of 3.5.5 and then a few more, even rolling upgrade issues have been fixed for 3.5.6. This is a more up-to-date guide: https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ

Re: upgrade from 3.4.5 to 3.5.6

Il giorno mar 24 mar 2020 alle ore 11:45 kuldeep singh ha scritto: > > Hi, > > Current Zookeeper version :- 3.4.5 > Upgraded version:- 3.5.6 > > We are not going with 3.5.7. Our final decision is zookeeper version is > 3.5.6 I suggest you to move to 3.5.7, 3.5.6 is an older

Re: upgrade from 3.4.5 to 3.5.6

Hi, Current Zookeeper version :- 3.4.5 Upgraded version:- 3.5.6 We are not going with 3.5.7. Our final decision is zookeeper version is 3.5.6 as per your reply first we need to move latest version of 3.4.x, like below 3.4.5 -> 3.4.14 -> 3.5.6 (Correct me if I am wrong here) But

Re: upgrade from 3.4.5 to 3.5.6

Hi You have to upgrade to latest 3.4.x Zookeeper then you will upgrade to 3.5.7. All should run well without issues Enrico Il Mar 24 Mar 2020, 10:18 kuldeep singh ha scritto: > Hi Team, > > We are upgrading zookeeper from 3.4.5 to 3.5.6. I have set up 3 node > cluster where 2 node are on

upgrade from 3.4.5 to 3.5.6

Hi Team, We are upgrading zookeeper from 3.4.5 to 3.5.6. I have set up 3 node cluster where 2 node are on 3.5.6 version and 1 node on 3.4.5. Everything is running fine and didn't get any issue on my system. but I found something on apache site that first we need to upgrade on 3.4.6 than we can