Hi, My ZK server is up and running in secure mode, But When I am trying to connect to the ZK server using ZKCli, it gives the below error.
WatchedEvent state:AuthFailed type:None path:null Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication provider for scheme: ztpasswd has x509 ip digest* Can someone please help me on this issue. we are using the 3.5.6 version. I appreciate if I will get a response as soon as possible, as I am stuck at this point. Thanks, --------------------- Kuldeep Singh Budania Software Architect On Thu, Jun 25, 2020 at 11:54 AM Enrico Olivelli - Diennea <enrico.olive...@diennea.com.invalid> wrote: > I mean in zoo.cfg > Not as a system property > > Enrico > > Il giorno 25/06/20, 08:19 "Enrico Olivelli - Diennea" < > enrico.olive...@diennea.com.INVALID> ha scritto: > > Hi > You have to enable Netty on the server side > > Something like: > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > Hope that helps > Enrico > > Il giorno 24/06/20, 19:17 "kuldeep singh" <kuldeep.sing...@gmail.com> > ha scritto: > > Hi, > > I got below error while setting SSL properties in zkEnv.sh > > > > > ========================================================================================== > > 2020-06-24 15:49:35,864 - INFO [main:QuorumPeerConfig@133] - > Reading > configuration from: /etc/zookeeper/zoo.cfg > > 2020-06-24 15:49:35,874 - INFO [main:QuorumPeerConfig@385] - > clientPortAddress is 0.0.0.0/0.0.0.0:10181 > > 2020-06-24 15:49:35,874 - INFO [main:QuorumPeerConfig@399] - > secureClientPortAddress is 0.0.0.0/0.0.0.0:2281 > > 2020-06-24 15:49:35,878 - INFO [main:X509Util@79] - Setting -D > jdk.tls.rejectClientInitiatedRenegotiation=true to disable > client-initiated > TLS renegotiation > > 2020-06-24 15:49:35,897 - INFO [main:DatadirCleanupManager@78] - > autopurge.snapRetainCount set to 3 > > 2020-06-24 15:49:35,897 - INFO [main:DatadirCleanupManager@79] - > autopurge.purgeInterval set to 1 > > 2020-06-24 15:49:35,898 - INFO [ > PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task > started. > > 2020-06-24 15:49:35,899 - INFO [main:ManagedUtil@46] - Log4j > found with > jmx enabled. > > 2020-06-24 15:49:35,903 - INFO [PurgeTask:FileTxnSnapLog@103] - > zookeeper.snapshot.trust.empty : false > > 2020-06-24 15:49:35,910 - INFO [ > PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge task > completed. > > 2020-06-24 15:49:35,975 - INFO [main:QuorumPeerMain@141] - > Starting quorum > peer > > 2020-06-24 15:49:35,983 - INFO [main:ServerCnxnFactory@135] - > Using > org.apache.zookeeper.server.NIOServerCnxnFactory as server > connection > factory > > 2020-06-24 15:49:35,986 - INFO [main:NIOServerCnxnFactory@673] - > Configuring NIO connection handler with 10s sessionless connection > timeout, > 2 selector thread(s), 16 worker threads, and 64 kB direct buffers. > > 2020-06-24 15:49:35,992 - INFO [main:NIOServerCnxnFactory@686] - > binding > to port 0.0.0.0/0.0.0.0:10181 > > 2020-06-24 15:49:35,994 - INFO [main:ServerCnxnFactory@135] - > Using > org.apache.zookeeper.server.NIOServerCnxnFactory as server > connection > factory > > 2020-06-24 15:49:35,995 - ERROR [main:QuorumPeerMain@101] - > Unexpected > exception, exiting abnormally > > java.lang.UnsupportedOperationException: SSL isn't supported in > NIOServerCnxn > > at > > org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:644) > > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:155) > > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) > > at > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) > > > > > > I have set the following properties in SERVER_JVMFLAGS in zkEnv.sh > file : > > "-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > -Dzookeeper.ssl.keyStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkkeystore.p12 > -Dzookeeper.ssl.keyStore.password=EvaiKiO1@123456 > > -Dzookeeper.ssl.trustStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkTrustStore.jks > -Dzookeeper.ssl.trustStore.password=EvaiKiO1@123456" > > Thanks, > --------------------- > Kuldeep Singh Budania > > > > On Mon, Jun 22, 2020 at 8:08 PM Jordan Zimmerman < > jor...@jordanzimmerman.com> > wrote: > > > It's the same as the normal ZooKeeper client: > > > https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions > > < > > > https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions > > > > > > > -Jordan > > > > > On Jun 22, 2020, at 5:50 AM, kuldeep singh < > kuldeep.sing...@gmail.com> > > wrote: > > > > > > Hi Team, > > > How we will do secure communication between the Curator > framework and > > > zookeeper 3.5.6 ? > > > I didn't get any solution right now. > > > I appreciate it if someone could help me with the same. > > > > > > Thanks, > > > --------------------- > > > Kuldeep Singh Budania > > > Software Architect > > > > > > > > > On Fri, Apr 17, 2020 at 4:53 PM Szalay-Bekő Máté < > > szalay.beko.m...@gmail.com> > > > wrote: > > > > > >> Hello Kuldeep, > > >> > > >> did you download the source from the ZooKeeper webpage, or > checked out > > from > > >> git? > > >> Anyway, the following commands should work: > > >> > > >> wget > > >> > > >> > > > https://downloads.apache.org/zookeeper/zookeeper-3.5.7/apache-zookeeper-3.5.7.tar.gz > > >> tar xzvf ./apache-zookeeper-3.5.7.tar.gz > > >> cd apache-zookeeper-3.5.7 > > >> mvn clean install -DskipTests > > >> > > >> I tested it with OpenJDK 8u424 and maven 3.6.0. > > >> > > >> Kind regards, > > >> Mate > > >> > > >> > > >> On Fri, Apr 17, 2020 at 12:51 PM kuldeep singh < > > kuldeep.sing...@gmail.com> > > >> wrote: > > >> > > >>> Hi All, > > >>> > > >>> I have download the zookeeper source code and run the mvn > command for > > >>> packaging but get problem on "zookeeper-server" directory. > > >>> > > >>> *command :- mvn clean package* > > >>> > > >>> [ERROR] Failed to execute goal > > >>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile > > >>> (default-compile) on project zookeeper: Fatal error > compiling: > > >>> java.lang.NullPointerException -> [Help 1] > > >>> org.apache.maven.lifecycle.LifecycleExecutionException: > Failed to > > execute > > >>> goal > org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile > > >>> (default-compile) on project zookeeper: Fatal error compiling > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) > > >>> at > > org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) > > >>> at > org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) > > >>> at > org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) > > >>> at > org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) > > >>> at > org.apache.maven.cli.MavenCli.main(MavenCli.java:157) > > >>> at > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > >>> at > > >>> > > >>> > > >> > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > >>> at > > >>> > > >>> > > >> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > >>> at java.lang.reflect.Method.invoke(Method.java:498) > > >>> at > > >>> > > >>> > > >> > > > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > > >>> at > > >>> > > >> > > > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > > >>> at > > >>> > > >>> > > >> > > > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > > >>> at > > >>> > > > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > > >>> Caused by: org.apache.maven.plugin.MojoExecutionException: > Fatal error > > >>> compiling > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1140) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:188) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) > > >>> ... 19 more > > >>> Caused by: org.codehaus.plexus.compiler.CompilerException: > > >>> java.lang.NullPointerException > > >>> at > > >>> > > >>> > > >> > > > org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:173) > > >>> at > > >>> > > >>> > > >> > > > org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174) > > >>> at > > >>> > > >>> > > >> > > > org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1129) > > >>> ... 22 more > > >>> Caused by: java.lang.RuntimeException: > java.lang.NullPointerException > > >>> at > com.sun.tools.javac.main.Main.compile(Main.java:559) > > >>> at > > >>> > com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) > > >>> at > > >>> > com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) > > >>> at > > >>> > > >>> > > >> > > > org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126) > > >>> ... 24 more > > >>> Caused by: java.lang.NullPointerException > > >>> at > > >>> > > >>> > > >> > > > com.sun.tools.javac.model.JavacTypes.getOverriddenMethods(JavacTypes.java:306) > > >>> at com.sun.tools.doclint.Env.setCurrent(Env.java:155) > > >>> at > com.sun.tools.doclint.Checker.scan(Checker.java:144) > > >>> at > com.sun.tools.doclint.DocLint$2.visitDecl(DocLint.java:285) > > >>> at > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:359) > > >>> at > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) > > >>> at > > >>> > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) > > >>> at > > >>> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> at > > >>> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > >>> at > > >>> > com.sun.source.util.TreeScanner.visitNewClass(TreeScanner.java:280) > > >>> at > > >>> > com.sun.tools.javac.tree.JCTree$JCNewClass.accept(JCTree.java:1532) > > >>> at > > >>> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> at > > >>> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > >>> at > > >>> > com.sun.source.util.TreeScanner.visitVariable(TreeScanner.java:153) > > >>> at > > >>> > > > com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:373) > > >>> at > > >>> > > > com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:346) > > >>> at > > >>> > com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:864) > > >>> at > > >>> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> at > > >>> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > >>> at > com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) > > >>> at > > >>> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) > > >>> at > > >> > com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:133) > > >>> at > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:360) > > >>> at > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) > > >>> at > > >>> > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) > > >>> at > > >>> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> at > com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) > > >>> at > > >>> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) > > >>> at > > >>> > > >> > > > com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:120) > > >>> at > > >>> > > >>> > > >> > > > com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:354) > > >>> at > > >>> > > >>> > > >> > > > com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:346) > > >>> at > > >>> > > > com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:550) > > >>> at > > >>> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> at > com.sun.tools.doclint.DocLint$3.started(DocLint.java:296) > > >>> at > > >>> > > >>> > > >> > > > com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.started(ClientCodeWrapper.java:668) > > >>> at > > >>> > > >>> > > >> > > > com.sun.tools.javac.api.MultiTaskListener.started(MultiTaskListener.java:103) > > >>> at > > >>> > com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1240) > > >>> at > > >>> > com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901) > > >>> at > > >>> > com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860) > > >>> at > com.sun.tools.javac.main.Main.compile(Main.java:523) > > >>> ... 27 more > > >>> [ERROR] > > >>> [ERROR] Re-run Maven using the -X switch to enable full > debug logging. > > >>> [ERROR] > > >>> [ERROR] For more information about the errors and possible > solutions, > > >>> please read the following articles: > > >>> [ERROR] [Help 1] > > >>> > > > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > > >>> > > >>> Can someone help me on this issue? > > >>> > > >>> Thanks, > > >>> --------------------- > > >>> Kuldeep Singh Budania > > >>> > > >>> > > >>> > > >>> On Sat, Apr 4, 2020 at 5:57 PM Szalay-Bekő Máté < > > >>> szalay.beko.m...@gmail.com> > > >>> wrote: > > >>> > > >>>> these exceptions can mean many things... I think this can > be even > > >> normal > > >>>> duding rolling restart (as some connections get broken in > this case) > > >>>> > > >>>> However, I saw cases already when exceptions like these > killed > > receiver > > >>> or > > >>>> sender threads in QuorumCnxManager / Leader Election in > such a way > > that > > >>>> they were not able to recover, so the node was unable to > connect to > > any > > >>>> quorum until restart. I remember seeing this in 3.4 too. > > >>>> > > >>>> Do you see these exceptions in the second server (the one > which you > > >> just > > >>>> upgraded in step 3)? > > >>>> Is this issue reproducible? > > >>>> > > >>>> What is the tickTime and initLimit you use? Maybe the > server just > > >> require > > >>>> more time to sync? > > >>>> > > >>>> I would need more logs to really see what happened. Can you > create a > > >> Jira > > >>>> issue and upload the logs and also the ZooKeeper configs? I > am happy > > to > > >>>> take a closer look. > > >>>> (if you need to re-run the test to collect the logs, then > enabling > > >> DEBUG > > >>>> logs would be great. The INFO level logs are usually enough > for these > > >>>> problems, but one can never know...) > > >>>> > > >>>> Kind regards, > > >>>> Mate > > >>>> > > >>>> > > >>>> On Fri, Apr 3, 2020 at 10:05 AM kuldeep singh < > > >> kuldeep.sing...@gmail.com > > >>>> > > >>>> wrote: > > >>>> > > >>>>> Hi Team, > > >>>>> > > >>>>> I have done some POC on rolling upgrade and found below > result. > > >>>>> > > >>>>> > > >>>>> 1. On 1st node upgrade zookeeper . Traffic was running > fine > > >> because > > >>> 2 > > >>>>> nodes are already on old zookeeper. > > >>>>> 2. On 1st node upgrade our application and didn’t find > any issue > > >>>>> 3. On 2nd node upgrade zookeeper but got below error and > zookeeper > > >>> is > > >>>>> not taking any requests > > >>>>> 4. > > >>>>> > > >>>>> java.io.EOFException > > >>>>> > > >>>>> at > java.io.DataInputStream.readInt(DataInputStream.java:392) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >>> > > >> > > > org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:747) > > >>>>> > > >>>>> 2020-03-30 14:19:55,587 - WARN > > >>>>> [RecvWorker:1:QuorumCnxManager$RecvWorker@765] - > Interrupting > > >>> SendWorker > > >>>>> > > >>>>> 2020-03-30 14:19:55,588 - ERROR [LearnerHandler-/ > 192.168.44.73:33754 > > >>>>> :LearnerHandler@562] - Unexpected exception causing > shutdown while > > >>> sock > > >>>>> still open > > >>>>> > > >>>>> java.io.EOFException > > >>>>> > > >>>>> at > java.io.DataInputStream.readInt(DataInputStream.java:392) > > >>>>> > > >>>>> at > > >>>>> > > >> > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >>> > > >> > > > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) > > >>>>> > > >>>>> at > > >>>>> > > >>>> > > >>> > > >> > > > org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108) > > >>>>> > > >>>>> at > > >>>>> > > >>>>> > > >>>> > > >>> > > >> > > > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:476) > > >>>>> > > >>>>> 2020-03-30 14:19:55,588 - WARN > > >>>>> [SendWorker:1:QuorumCnxManager$SendWorker@679] - > Interrupted while > > >>>> waiting > > >>>>> for message on queue > > >>>>> > > >>>>> Please let me know is this the known issue or this is > different issue > > >>>> which > > >>>>> is mention in Apache zookeeper documentation when > upgrading from > > >> 3.4.5 > > >>> to > > >>>>> 3.5.6 > > >>>>> > > >>>>> Thanks, > > >>>>> --------------------- > > >>>>> Kuldeep Singh Budania > > >>>>> Software Architect > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Sun, Mar 29, 2020 at 9:06 AM Alexander Shraer < > shra...@gmail.com> > > >>>>> wrote: > > >>>>> > > >>>>>> +1 to what Mate said (I wrote the quoted instructions). > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>> On Tue, Mar 24, 2020 at 7:03 AM Szalay-Bekő Máté < > > >>>>>> szalay.beko.m...@gmail.com> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> Hi Kuldeep, > > >>>>>>> > > >>>>>>> I just want to provide you some background info about our > > >>>>> documentation. > > >>>>>>> The reason to upgrade to 3.4.6 first is to avoid the > following > > >>> error: > > >>>>>>> > > >>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN [localhost/ > > >>> 127.0.0.1:2784 > > >>>>>>> :QuorumCnxManager@349] - Invalid server id: -65536 > > >>>>>>> > > >>>>>>> This error comes because of the protocol changes between > > >> ZooKeeper > > >>>>> server > > >>>>>>> nodes during connection initiation for leader election. > In > > >>> ZooKeeper > > >>>>> 3.5 > > >>>>>> a > > >>>>>>> protocol version was introduced (see ZOOKEEPER-107) and > since > > >> that > > >>>> time > > >>>>>> the > > >>>>>>> fist long value sent in the initial message is not the > server ID > > >>> but > > >>>>> the > > >>>>>>> protocol version (-65536). In ZooKeeper 3.4.6 we made > the old 3.4 > > >>>>>>> ZooKeepers backward compatible, so they are able to > parse both > > >> the > > >>>> old > > >>>>>> and > > >>>>>>> the new protocol format (see ZOOKEEPER-1633). This issue > happens > > >>> only > > >>>>>> when > > >>>>>>> you need to use old (3.4.0 - 3.4.5) and new (3.5.0+) > ZooKeeper > > >>>> servers > > >>>>>>> together in the same cluster. During a rolling upgrade, > this is > > >>>> usually > > >>>>>> the > > >>>>>>> case to have old and new ZooKeepers present together. > > >>>>>>> > > >>>>>>> The fact that you haven't seen any issues might be > caused by the > > >>>> order > > >>>>> of > > >>>>>>> the servers. In ZooKeeper the connection initiation > between the > > >>>> servers > > >>>>>>> during the leader election follows a specific rule. As > far as I > > >>>>> remember > > >>>>>>> always the server with the larger ID 'wins the > challenge', so it > > >> is > > >>>>>>> possible, that the old server didn't need to parse any > initial > > >>>> message > > >>>>>> (if > > >>>>>>> it had the largest ID) and this is why you haven't seen > the > > >> issue. > > >>>> Also > > >>>>>>> having 2 nodes up from the 3 nodes cluster still makes > the > > >> cluster > > >>>> work > > >>>>>> (so > > >>>>>>> you should also check if all the servers are part of the > quorum). > > >>>>>>> > > >>>>>>> I agree with Enrico and Norbert, the safest and most > stable way > > >> is > > >>>>>> upgrade > > >>>>>>> first to 3.4.latest, then go to 3.5.latest. Still, if > you don't > > >> see > > >>>>> that > > >>>>>>> you would hit this specific issue (e.g. no "Invalid > server id" in > > >>> the > > >>>>> log > > >>>>>>> files), and all the three servers can handle traffic, > then maybe > > >>> you > > >>>>>> don't > > >>>>>>> need to upgrade first to 3.4.latest, it is your decision. > > >>> Definitely > > >>>>> you > > >>>>>>> should test it first, as suggested by the others. > > >>>>>>> > > >>>>>>> Kind regards, > > >>>>>>> Mate > > >>>>>>> > > >>>>>>> On Tue, Mar 24, 2020 at 12:29 PM Norbert Kalmar > > >>>>>>> <nkal...@cloudera.com.invalid> wrote: > > >>>>>>> > > >>>>>>>> Hi, > > >>>>>>>> > > >>>>>>>> That guide is to upgrade to 3.5.0, which was an alpha > version. > > >> A > > >>>> lot > > >>>>>> has > > >>>>>>>> changed for the first stable release of 3.5.5 and then > a few > > >>> more, > > >>>>> even > > >>>>>>>> rolling upgrade issues have been fixed for 3.5.6. > > >>>>>>>> This is a more up-to-date guide: > > >>>>>>>> > > >>> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ > > >>>>>>>> > > >>>>>>>> If you have done your testing (with prod snapshot!), > then you > > >> can > > >>>>> skip > > >>>>>>> 3.4 > > >>>>>>>> latest upgrade, but keep in mind we do our > recommendations for > > >> a > > >>>>>> reason. > > >>>>>>>> There were issues reported and/or found during testing. > Some > > >> are > > >>>>> fixed > > >>>>>>> with > > >>>>>>>> 3.5.6, some only happens if certain conditions stand > > >>> (IOException: > > >>>> No > > >>>>>>>> snapshot found - mentioned in the guide, fixed in > 3.5.6). > > >>>>>>>> > > >>>>>>>> So it is up to you, I would still recommend to do an 3.4 > > >> upgrade > > >>>>> first, > > >>>>>>> if > > >>>>>>>> it's feasible. > > >>>>>>>> > > >>>>>>>> Regards, > > >>>>>>>> Norbert > > >>>>>>>> > > >>>>>>>> On Tue, Mar 24, 2020 at 11:45 AM kuldeep singh < > > >>>>>>> kuldeep.sing...@gmail.com> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Hi, > > >>>>>>>>> > > >>>>>>>>> Current Zookeeper version :- 3.4.5 > > >>>>>>>>> Upgraded version :- 3.5.6 > > >>>>>>>>> > > >>>>>>>>> We are not going with 3.5.7. Our final decision is > zookeeper > > >>>>> version > > >>>>>> is > > >>>>>>>>> 3.5.6 > > >>>>>>>>> as per your reply first we need to move latest version > of > > >>> 3.4.x, > > >>>>> like > > >>>>>>>> below > > >>>>>>>>> > > >>>>>>>>> 3.4.5 -> 3.4.14 -> 3.5.6 (Correct me if I am wrong > here) > > >>>>>>>>> > > >>>>>>>>> But if We are not facing any problem that i have > shared you > > >>> that > > >>>> we > > >>>>>>> have > > >>>>>>>>> set up of 3 node cluster where 2 node are on 3.5.6 > version > > >> and > > >>> 1 > > >>>>> node > > >>>>>>> on > > >>>>>>>>> 3.4.5, Everything is running fine and didn't get any > issue, > > >> So > > >>>> what > > >>>>>>> other > > >>>>>>>>> problem we can face if we directly move to 3.5.6 > > >>>>>>>>> > > >>>>>>>>> Thanks, > > >>>>>>>>> --------------------- > > >>>>>>>>> Kuldeep Singh Budania > > >>>>>>>>> Software Architect > > >>>>>>>>> > > >>>>>>>>> > > >>>>>>>>> On Tue, Mar 24, 2020 at 3:58 PM Enrico Olivelli < > > >>>>> eolive...@gmail.com > > >>>>>>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Hi > > >>>>>>>>>> You have to upgrade to latest 3.4.x Zookeeper then > you will > > >>>>> upgrade > > >>>>>>> to > > >>>>>>>>>> 3.5.7. > > >>>>>>>>>> All should run well without issues > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Enrico > > >>>>>>>>>> > > >>>>>>>>>> Il Mar 24 Mar 2020, 10:18 kuldeep singh < > > >>>>> kuldeep.sing...@gmail.com > > >>>>>>> > > >>>>>>> ha > > >>>>>>>>>> scritto: > > >>>>>>>>>> > > >>>>>>>>>>> Hi Team, > > >>>>>>>>>>> > > >>>>>>>>>>> We are upgrading zookeeper from 3.4.5 to 3.5.6. I > have > > >> set > > >>>> up 3 > > >>>>>>> node > > >>>>>>>>>>> cluster where 2 node are on 3.5.6 version and 1 node > on > > >>>> 3.4.5. > > >>>>>>>>>>> > > >>>>>>>>>>> Everything is running fine and didn't get any issue > on my > > >>>>> system. > > >>>>>>>>>>> > > >>>>>>>>>>> but I found something on apache site that first we > need > > >> to > > >>>>>> upgrade > > >>>>>>>> on > > >>>>>>>>>>> 3.4.6 than we can upgrade to 3.5.6. So is it > mandatory > > >> to > > >>> go > > >>>>> on > > >>>>>>>> 3.4.6 > > >>>>>>>>>>> first. > > >>>>>>>>>>> > > >>>>>>>>>>> *Upgrading to 3.5.0* > > >>>>>>>>>>> > > >>>>>>>>>>> Upgrading a running ZooKeeper ensemble to 3.5.0 > should be > > >>>> done > > >>>>>> only > > >>>>>>>>> after > > >>>>>>>>>>> upgrading your ensemble to the 3.4.6 release. Note > that > > >>> this > > >>>> is > > >>>>>>> only > > >>>>>>>>>>> necessary for rolling upgrades (if you're fine with > > >>> shutting > > >>>>> down > > >>>>>>> the > > >>>>>>>>>>> system completely, you don't have to go through > 3.4.6). > > >> If > > >>>> you > > >>>>>>>> attempt > > >>>>>>>>> a > > >>>>>>>>>>> rolling upgrade without going through 3.4.6 (for > example > > >>> from > > >>>>>>> 3.4.5), > > >>>>>>>>> you > > >>>>>>>>>>> may get the following error: > > >>>>>>>>>>> > > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - INFO [localhost/ > > >>>>>> 127.0.0.1:2784 > > >>>>>>>>>>> :QuorumCnxManager$Listener@498] - Received > connection > > >>>> request > > >>>>> / > > >>>>>>>>>>> 127.0.0.1:60876 > > >>>>>>>>>>> > > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN [localhost/ > > >>>>>> 127.0.0.1:2784 > > >>>>>>>>>>> :QuorumCnxManager@349] - Invalid server id: -65536 > > >>>>>>>>>>> > > >>>>>>>>>>> During a rolling upgrade, each server is taken down > in > > >> turn > > >>>> and > > >>>>>>>>> rebooted > > >>>>>>>>>>> with the new 3.5.0 binaries. Before starting the > server > > >>> with > > >>>>>> 3.5.0 > > >>>>>>>>>>> binaries, we highly recommend updating the > configuration > > >>> file > > >>>>> so > > >>>>>>> that > > >>>>>>>>> all > > >>>>>>>>>>> server statements "server.x=..." contain client ports > > >> (see > > >>>> the > > >>>>>>>> section > > >>>>>>>>>>> Specifying > > >>>>>>>>>>> the client port). As explained earlier you may leave > the > > >>>>>>>> configuration > > >>>>>>>>>> in a > > >>>>>>>>>>> single file, as well as leave the > > >>>> clientPort/clientPortAddress > > >>>>>>>>> statements > > >>>>>>>>>>> (although if you specify client ports in the new > format, > > >>>> these > > >>>>>>>>> statements > > >>>>>>>>>>> are now redundant). > > >>>>>>>>>>> > > >>>>>>>>>>> Could you please let me know about this case. > Appreciate > > >> if > > >>>>>> respond > > >>>>>>>>> soon. > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks, > > >>>>>>>>>>> --------------------- > > >>>>>>>>>>> Kuldeep Singh Budania > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > ________________________________ > > CONFIDENTIALITY & PRIVACY NOTICE > This e-mail (including any attachments) is strictly confidential and > may also contain privileged information. If you are not the intended > recipient you are not authorised to read, print, save, process or disclose > this message. If you have received this message by mistake, please inform > the sender immediately and destroy this e-mail, its attachments and any > copies. Any use, distribution, reproduction or disclosure by any person > other than the intended recipient is strictly prohibited and the person > responsible may incur in penalties. > The use of this e-mail is only for professional purposes; there is no > guarantee that the correspondence towards this e-mail will be read only by > the recipient, because, under certain circumstances, there may be a need to > access this email by third subjects belonging to the Company. > > > ________________________________ > > CONFIDENTIALITY & PRIVACY NOTICE > This e-mail (including any attachments) is strictly confidential and may > also contain privileged information. If you are not the intended recipient > you are not authorised to read, print, save, process or disclose this > message. If you have received this message by mistake, please inform the > sender immediately and destroy this e-mail, its attachments and any copies. > Any use, distribution, reproduction or disclosure by any person other than > the intended recipient is strictly prohibited and the person responsible > may incur in penalties. > The use of this e-mail is only for professional purposes; there is no > guarantee that the correspondence towards this e-mail will be read only by > the recipient, because, under certain circumstances, there may be a need to > access this email by third subjects belonging to the Company. >
