> No authentication provider for scheme: ztpasswd has x509 ip digest* This suggest you have some configuration error... Where did you use the "ztpasswd" string in your configs / commands?
On Wed, Jul 1, 2020 at 2:53 PM Szalay-Bekő Máté <szalay.beko.m...@gmail.com> wrote: > > My ZK server is up and running in secure mode > > What is your goal? You want to setup client-server SSL connection? > > see: > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > > (or you want to have both SSL and SASL enabled?) > > Anyway, please remove the following line from the SERVER_JVMFLAGS: > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > This is a configuration that makes sense only for the ZooKeeper client, > not for the server. For the server, use the following: > > -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > If it doesn't solve the issue, then can you please send your zoo.cfg file? > Also can you please send the zkCli command you execute? (you need to > connect to the secure ZooKeeper port, unless portUnification is enabled) > > Kind regards, > Mate > > On Wed, Jul 1, 2020 at 9:48 AM kuldeep singh <kuldeep.sing...@gmail.com> > wrote: > >> Hi, >> >> we have done below changes in java.env file >> >> export SERVER_JVMFLAGS=" >> >> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >> -Dzookeeper.ssl.keyStore.password=testpass >> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >> -Dzookeeper.ssl.trustStore.password=testpass" >> >> >> >> export CLIENT_JVMFLAGS=" >> >> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >> -Dzookeeper.client.secure=true >> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >> -Dzookeeper.ssl.keyStore.password=testpass >> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >> -Dzookeeper.ssl.trustStore.password=testpass" >> >> I have started the ZK server and it is up without any issue. >> >> But now when I login to ZkCli then it gives the below error. >> >> WatchedEvent state:AuthFailed type:None path:null >> >> >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication provider >> for scheme: ztpasswd has x509 ip digest* >> >> Please help me on this issue >> >> Thanks, >> --------------------- >> Kuldeep Singh Budania >> Software Architect >> >> >> On Wed, Jul 1, 2020 at 12:05 PM kuldeep singh <kuldeep.sing...@gmail.com> >> wrote: >> >> > Hi, >> > >> > My ZK server is up and running in secure mode, But When I am trying to >> > connect to the ZK server using ZKCli, it gives the below error. >> > >> > WatchedEvent state:AuthFailed type:None path:null >> > >> > >> > Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN >> > [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication >> provider >> > for scheme: ztpasswd has x509 ip digest* >> > >> > Can someone please help me on this issue. we are using the 3.5.6 >> version. >> > >> > I appreciate if I will get a response as soon as possible, as I am stuck >> > at this point. >> > >> > Thanks, >> > --------------------- >> > Kuldeep Singh Budania >> > Software Architect >> > >> > >> > >> > On Thu, Jun 25, 2020 at 11:54 AM Enrico Olivelli - Diennea >> > <enrico.olive...@diennea.com.invalid> wrote: >> > >> >> I mean in zoo.cfg >> >> Not as a system property >> >> >> >> Enrico >> >> >> >> Il giorno 25/06/20, 08:19 "Enrico Olivelli - Diennea" < >> >> enrico.olive...@diennea.com.INVALID> ha scritto: >> >> >> >> Hi >> >> You have to enable Netty on the server side >> >> >> >> Something like: >> >> >> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> >> >> >> Hope that helps >> >> Enrico >> >> >> >> Il giorno 24/06/20, 19:17 "kuldeep singh" < >> kuldeep.sing...@gmail.com> >> >> ha scritto: >> >> >> >> Hi, >> >> >> >> I got below error while setting SSL properties in zkEnv.sh >> >> >> >> >> >> >> >> >> >> >> ========================================================================================== >> >> >> >> 2020-06-24 15:49:35,864 - INFO [main:QuorumPeerConfig@133] - >> >> Reading >> >> configuration from: /etc/zookeeper/zoo.cfg >> >> >> >> 2020-06-24 15:49:35,874 - INFO [main:QuorumPeerConfig@385] - >> >> clientPortAddress is 0.0.0.0/0.0.0.0:10181 >> >> >> >> 2020-06-24 15:49:35,874 - INFO [main:QuorumPeerConfig@399] - >> >> secureClientPortAddress is 0.0.0.0/0.0.0.0:2281 >> >> >> >> 2020-06-24 15:49:35,878 - INFO [main:X509Util@79] - Setting >> -D >> >> jdk.tls.rejectClientInitiatedRenegotiation=true to disable >> >> client-initiated >> >> TLS renegotiation >> >> >> >> 2020-06-24 15:49:35,897 - INFO [main:DatadirCleanupManager@78] >> - >> >> autopurge.snapRetainCount set to 3 >> >> >> >> 2020-06-24 15:49:35,897 - INFO [main:DatadirCleanupManager@79] >> - >> >> autopurge.purgeInterval set to 1 >> >> >> >> 2020-06-24 15:49:35,898 - INFO [ >> >> PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge task >> >> started. >> >> >> >> 2020-06-24 15:49:35,899 - INFO [main:ManagedUtil@46] - Log4j >> >> found with >> >> jmx enabled. >> >> >> >> 2020-06-24 15:49:35,903 - INFO [PurgeTask:FileTxnSnapLog@103] >> - >> >> zookeeper.snapshot.trust.empty : false >> >> >> >> 2020-06-24 15:49:35,910 - INFO [ >> >> PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge task >> >> completed. >> >> >> >> 2020-06-24 15:49:35,975 - INFO [main:QuorumPeerMain@141] - >> >> Starting quorum >> >> peer >> >> >> >> 2020-06-24 15:49:35,983 - INFO [main:ServerCnxnFactory@135] - >> >> Using >> >> org.apache.zookeeper.server.NIOServerCnxnFactory as server >> >> connection >> >> factory >> >> >> >> 2020-06-24 15:49:35,986 - INFO [main:NIOServerCnxnFactory@673] >> - >> >> Configuring NIO connection handler with 10s sessionless >> >> connection timeout, >> >> 2 selector thread(s), 16 worker threads, and 64 kB direct >> buffers. >> >> >> >> 2020-06-24 15:49:35,992 - INFO [main:NIOServerCnxnFactory@686 >> ] >> >> - binding >> >> to port 0.0.0.0/0.0.0.0:10181 >> >> >> >> 2020-06-24 15:49:35,994 - INFO [main:ServerCnxnFactory@135] - >> >> Using >> >> org.apache.zookeeper.server.NIOServerCnxnFactory as server >> >> connection >> >> factory >> >> >> >> 2020-06-24 15:49:35,995 - ERROR [main:QuorumPeerMain@101] - >> >> Unexpected >> >> exception, exiting abnormally >> >> >> >> java.lang.UnsupportedOperationException: SSL isn't supported in >> >> NIOServerCnxn >> >> >> >> at >> >> >> >> >> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:644) >> >> >> >> at >> >> >> >> >> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:155) >> >> >> >> at >> >> >> >> >> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) >> >> >> >> at >> >> >> >> >> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) >> >> >> >> >> >> >> >> >> >> >> >> I have set the following properties in SERVER_JVMFLAGS in >> >> zkEnv.sh file : >> >> >> >> >> "-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> >> >> >> >> -Dzookeeper.ssl.keyStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkkeystore.p12 >> >> -Dzookeeper.ssl.keyStore.password=EvaiKiO1@123456 >> >> >> >> >> -Dzookeeper.ssl.trustStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkTrustStore.jks >> >> -Dzookeeper.ssl.trustStore.password=EvaiKiO1@123456" >> >> >> >> Thanks, >> >> --------------------- >> >> Kuldeep Singh Budania >> >> >> >> >> >> >> >> On Mon, Jun 22, 2020 at 8:08 PM Jordan Zimmerman < >> >> jor...@jordanzimmerman.com> >> >> wrote: >> >> >> >> > It's the same as the normal ZooKeeper client: >> >> > >> >> >> https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions >> >> > < >> >> > >> >> >> https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions >> >> > > >> >> > >> >> > -Jordan >> >> > >> >> > > On Jun 22, 2020, at 5:50 AM, kuldeep singh < >> >> kuldeep.sing...@gmail.com> >> >> > wrote: >> >> > > >> >> > > Hi Team, >> >> > > How we will do secure communication between the Curator >> >> framework and >> >> > > zookeeper 3.5.6 ? >> >> > > I didn't get any solution right now. >> >> > > I appreciate it if someone could help me with the same. >> >> > > >> >> > > Thanks, >> >> > > --------------------- >> >> > > Kuldeep Singh Budania >> >> > > Software Architect >> >> > > >> >> > > >> >> > > On Fri, Apr 17, 2020 at 4:53 PM Szalay-Bekő Máté < >> >> > szalay.beko.m...@gmail.com> >> >> > > wrote: >> >> > > >> >> > >> Hello Kuldeep, >> >> > >> >> >> > >> did you download the source from the ZooKeeper webpage, or >> >> checked out >> >> > from >> >> > >> git? >> >> > >> Anyway, the following commands should work: >> >> > >> >> >> > >> wget >> >> > >> >> >> > >> >> >> > >> >> >> https://downloads.apache.org/zookeeper/zookeeper-3.5.7/apache-zookeeper-3.5.7.tar.gz >> >> > >> tar xzvf ./apache-zookeeper-3.5.7.tar.gz >> >> > >> cd apache-zookeeper-3.5.7 >> >> > >> mvn clean install -DskipTests >> >> > >> >> >> > >> I tested it with OpenJDK 8u424 and maven 3.6.0. >> >> > >> >> >> > >> Kind regards, >> >> > >> Mate >> >> > >> >> >> > >> >> >> > >> On Fri, Apr 17, 2020 at 12:51 PM kuldeep singh < >> >> > kuldeep.sing...@gmail.com> >> >> > >> wrote: >> >> > >> >> >> > >>> Hi All, >> >> > >>> >> >> > >>> I have download the zookeeper source code and run the mvn >> >> command for >> >> > >>> packaging but get problem on "zookeeper-server" >> directory. >> >> > >>> >> >> > >>> *command :- mvn clean package* >> >> > >>> >> >> > >>> [ERROR] Failed to execute goal >> >> > >>> >> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile >> >> > >>> (default-compile) on project zookeeper: Fatal error >> >> compiling: >> >> > >>> java.lang.NullPointerException -> [Help 1] >> >> > >>> org.apache.maven.lifecycle.LifecycleExecutionException: >> >> Failed to >> >> > execute >> >> > >>> goal >> >> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile >> >> > >>> (default-compile) on project zookeeper: Fatal error >> >> compiling >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) >> >> > >>> at >> >> > >> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) >> >> > >>> at >> >> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) >> >> > >>> at >> >> org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) >> >> > >>> at >> >> org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) >> >> > >>> at >> >> org.apache.maven.cli.MavenCli.main(MavenCli.java:157) >> >> > >>> at >> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> > >>> at >> java.lang.reflect.Method.invoke(Method.java:498) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) >> >> > >>> at >> >> > >>> >> >> > >> >> >> > >> >> >> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) >> >> > >>> at >> >> > >>> >> >> > >> >> >> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) >> >> > >>> Caused by: >> org.apache.maven.plugin.MojoExecutionException: >> >> Fatal error >> >> > >>> compiling >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1140) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:188) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) >> >> > >>> ... 19 more >> >> > >>> Caused by: >> org.codehaus.plexus.compiler.CompilerException: >> >> > >>> java.lang.NullPointerException >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:173) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1129) >> >> > >>> ... 22 more >> >> > >>> Caused by: java.lang.RuntimeException: >> >> java.lang.NullPointerException >> >> > >>> at >> >> com.sun.tools.javac.main.Main.compile(Main.java:559) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126) >> >> > >>> ... 24 more >> >> > >>> Caused by: java.lang.NullPointerException >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> com.sun.tools.javac.model.JavacTypes.getOverriddenMethods(JavacTypes.java:306) >> >> > >>> at >> com.sun.tools.doclint.Env.setCurrent(Env.java:155) >> >> > >>> at >> >> com.sun.tools.doclint.Checker.scan(Checker.java:144) >> >> > >>> at >> >> com.sun.tools.doclint.DocLint$2.visitDecl(DocLint.java:285) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:359) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.visitNewClass(TreeScanner.java:280) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.tree.JCTree$JCNewClass.accept(JCTree.java:1532) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.visitVariable(TreeScanner.java:153) >> >> > >>> at >> >> > >>> >> >> > >> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:373) >> >> > >>> at >> >> > >>> >> >> > >> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:346) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:864) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) >> >> > >>> at >> >> com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) >> >> > >>> at >> >> > >> >> >> com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:133) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:360) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> >> > >>> at >> >> com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) >> >> > >>> at >> >> > >>> >> >> > >> >> >> > >> >> >> com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:120) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:354) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:346) >> >> > >>> at >> >> > >>> >> >> > >> >> >> com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:550) >> >> > >>> at >> >> > >>> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> >> > >>> at >> >> com.sun.tools.doclint.DocLint$3.started(DocLint.java:296) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.started(ClientCodeWrapper.java:668) >> >> > >>> at >> >> > >>> >> >> > >>> >> >> > >> >> >> > >> >> >> com.sun.tools.javac.api.MultiTaskListener.started(MultiTaskListener.java:103) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1240) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901) >> >> > >>> at >> >> > >>> >> >> com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860) >> >> > >>> at >> >> com.sun.tools.javac.main.Main.compile(Main.java:523) >> >> > >>> ... 27 more >> >> > >>> [ERROR] >> >> > >>> [ERROR] Re-run Maven using the -X switch to enable full >> >> debug logging. >> >> > >>> [ERROR] >> >> > >>> [ERROR] For more information about the errors and >> possible >> >> solutions, >> >> > >>> please read the following articles: >> >> > >>> [ERROR] [Help 1] >> >> > >>> >> >> > >> >> >> http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException >> >> > >>> >> >> > >>> Can someone help me on this issue? >> >> > >>> >> >> > >>> Thanks, >> >> > >>> --------------------- >> >> > >>> Kuldeep Singh Budania >> >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> On Sat, Apr 4, 2020 at 5:57 PM Szalay-Bekő Máté < >> >> > >>> szalay.beko.m...@gmail.com> >> >> > >>> wrote: >> >> > >>> >> >> > >>>> these exceptions can mean many things... I think this >> can >> >> be even >> >> > >> normal >> >> > >>>> duding rolling restart (as some connections get broken >> in >> >> this case) >> >> > >>>> >> >> > >>>> However, I saw cases already when exceptions like these >> >> killed >> >> > receiver >> >> > >>> or >> >> > >>>> sender threads in QuorumCnxManager / Leader Election in >> >> such a way >> >> > that >> >> > >>>> they were not able to recover, so the node was unable to >> >> connect to >> >> > any >> >> > >>>> quorum until restart. I remember seeing this in 3.4 too. >> >> > >>>> >> >> > >>>> Do you see these exceptions in the second server (the >> one >> >> which you >> >> > >> just >> >> > >>>> upgraded in step 3)? >> >> > >>>> Is this issue reproducible? >> >> > >>>> >> >> > >>>> What is the tickTime and initLimit you use? Maybe the >> >> server just >> >> > >> require >> >> > >>>> more time to sync? >> >> > >>>> >> >> > >>>> I would need more logs to really see what happened. Can >> >> you create a >> >> > >> Jira >> >> > >>>> issue and upload the logs and also the ZooKeeper >> configs? >> >> I am happy >> >> > to >> >> > >>>> take a closer look. >> >> > >>>> (if you need to re-run the test to collect the logs, >> then >> >> enabling >> >> > >> DEBUG >> >> > >>>> logs would be great. The INFO level logs are usually >> >> enough for these >> >> > >>>> problems, but one can never know...) >> >> > >>>> >> >> > >>>> Kind regards, >> >> > >>>> Mate >> >> > >>>> >> >> > >>>> >> >> > >>>> On Fri, Apr 3, 2020 at 10:05 AM kuldeep singh < >> >> > >> kuldeep.sing...@gmail.com >> >> > >>>> >> >> > >>>> wrote: >> >> > >>>> >> >> > >>>>> Hi Team, >> >> > >>>>> >> >> > >>>>> I have done some POC on rolling upgrade and found below >> >> result. >> >> > >>>>> >> >> > >>>>> >> >> > >>>>> 1. On 1st node upgrade zookeeper . Traffic was >> running >> >> fine >> >> > >> because >> >> > >>> 2 >> >> > >>>>> nodes are already on old zookeeper. >> >> > >>>>> 2. On 1st node upgrade our application and didn’t >> find >> >> any issue >> >> > >>>>> 3. On 2nd node upgrade zookeeper but got below error >> >> and zookeeper >> >> > >>> is >> >> > >>>>> not taking any requests >> >> > >>>>> 4. >> >> > >>>>> >> >> > >>>>> java.io.EOFException >> >> > >>>>> >> >> > >>>>> at >> >> java.io.DataInputStream.readInt(DataInputStream.java:392) >> >> > >>>>> >> >> > >>>>> at >> >> > >>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:747) >> >> > >>>>> >> >> > >>>>> 2020-03-30 14:19:55,587 - WARN >> >> > >>>>> [RecvWorker:1:QuorumCnxManager$RecvWorker@765] - >> >> Interrupting >> >> > >>> SendWorker >> >> > >>>>> >> >> > >>>>> 2020-03-30 14:19:55,588 - ERROR [LearnerHandler-/ >> >> 192.168.44.73:33754 >> >> > >>>>> :LearnerHandler@562] - Unexpected exception causing >> >> shutdown while >> >> > >>> sock >> >> > >>>>> still open >> >> > >>>>> >> >> > >>>>> java.io.EOFException >> >> > >>>>> >> >> > >>>>> at >> >> java.io.DataInputStream.readInt(DataInputStream.java:392) >> >> > >>>>> >> >> > >>>>> at >> >> > >>>>> >> >> > >> >> >> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) >> >> > >>>>> >> >> > >>>>> at >> >> > >>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) >> >> > >>>>> >> >> > >>>>> at >> >> > >>>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108) >> >> > >>>>> >> >> > >>>>> at >> >> > >>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> >> org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:476) >> >> > >>>>> >> >> > >>>>> 2020-03-30 14:19:55,588 - WARN >> >> > >>>>> [SendWorker:1:QuorumCnxManager$SendWorker@679] - >> >> Interrupted while >> >> > >>>> waiting >> >> > >>>>> for message on queue >> >> > >>>>> >> >> > >>>>> Please let me know is this the known issue or this is >> >> different issue >> >> > >>>> which >> >> > >>>>> is mention in Apache zookeeper documentation when >> >> upgrading from >> >> > >> 3.4.5 >> >> > >>> to >> >> > >>>>> 3.5.6 >> >> > >>>>> >> >> > >>>>> Thanks, >> >> > >>>>> --------------------- >> >> > >>>>> Kuldeep Singh Budania >> >> > >>>>> Software Architect >> >> > >>>>> >> >> > >>>>> >> >> > >>>>> >> >> > >>>>> On Sun, Mar 29, 2020 at 9:06 AM Alexander Shraer < >> >> shra...@gmail.com> >> >> > >>>>> wrote: >> >> > >>>>> >> >> > >>>>>> +1 to what Mate said (I wrote the quoted >> instructions). >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>>> >> >> > >>>>>> On Tue, Mar 24, 2020 at 7:03 AM Szalay-Bekő Máté < >> >> > >>>>>> szalay.beko.m...@gmail.com> >> >> > >>>>>> wrote: >> >> > >>>>>> >> >> > >>>>>>> Hi Kuldeep, >> >> > >>>>>>> >> >> > >>>>>>> I just want to provide you some background info about >> >> our >> >> > >>>>> documentation. >> >> > >>>>>>> The reason to upgrade to 3.4.6 first is to avoid the >> >> following >> >> > >>> error: >> >> > >>>>>>> >> >> > >>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN [localhost/ >> >> > >>> 127.0.0.1:2784 >> >> > >>>>>>> :QuorumCnxManager@349] - Invalid server id: -65536 >> >> > >>>>>>> >> >> > >>>>>>> This error comes because of the protocol changes >> between >> >> > >> ZooKeeper >> >> > >>>>> server >> >> > >>>>>>> nodes during connection initiation for leader >> election. >> >> In >> >> > >>> ZooKeeper >> >> > >>>>> 3.5 >> >> > >>>>>> a >> >> > >>>>>>> protocol version was introduced (see ZOOKEEPER-107) >> and >> >> since >> >> > >> that >> >> > >>>> time >> >> > >>>>>> the >> >> > >>>>>>> fist long value sent in the initial message is not >> the >> >> server ID >> >> > >>> but >> >> > >>>>> the >> >> > >>>>>>> protocol version (-65536). In ZooKeeper 3.4.6 we made >> >> the old 3.4 >> >> > >>>>>>> ZooKeepers backward compatible, so they are able to >> >> parse both >> >> > >> the >> >> > >>>> old >> >> > >>>>>> and >> >> > >>>>>>> the new protocol format (see ZOOKEEPER-1633). This >> >> issue happens >> >> > >>> only >> >> > >>>>>> when >> >> > >>>>>>> you need to use old (3.4.0 - 3.4.5) and new (3.5.0+) >> >> ZooKeeper >> >> > >>>> servers >> >> > >>>>>>> together in the same cluster. During a rolling >> upgrade, >> >> this is >> >> > >>>> usually >> >> > >>>>>> the >> >> > >>>>>>> case to have old and new ZooKeepers present together. >> >> > >>>>>>> >> >> > >>>>>>> The fact that you haven't seen any issues might be >> >> caused by the >> >> > >>>> order >> >> > >>>>> of >> >> > >>>>>>> the servers. In ZooKeeper the connection initiation >> >> between the >> >> > >>>> servers >> >> > >>>>>>> during the leader election follows a specific rule. >> As >> >> far as I >> >> > >>>>> remember >> >> > >>>>>>> always the server with the larger ID 'wins the >> >> challenge', so it >> >> > >> is >> >> > >>>>>>> possible, that the old server didn't need to parse >> any >> >> initial >> >> > >>>> message >> >> > >>>>>> (if >> >> > >>>>>>> it had the largest ID) and this is why you haven't >> seen >> >> the >> >> > >> issue. >> >> > >>>> Also >> >> > >>>>>>> having 2 nodes up from the 3 nodes cluster still >> makes >> >> the >> >> > >> cluster >> >> > >>>> work >> >> > >>>>>> (so >> >> > >>>>>>> you should also check if all the servers are part of >> >> the quorum). >> >> > >>>>>>> >> >> > >>>>>>> I agree with Enrico and Norbert, the safest and most >> >> stable way >> >> > >> is >> >> > >>>>>> upgrade >> >> > >>>>>>> first to 3.4.latest, then go to 3.5.latest. Still, if >> >> you don't >> >> > >> see >> >> > >>>>> that >> >> > >>>>>>> you would hit this specific issue (e.g. no "Invalid >> >> server id" in >> >> > >>> the >> >> > >>>>> log >> >> > >>>>>>> files), and all the three servers can handle traffic, >> >> then maybe >> >> > >>> you >> >> > >>>>>> don't >> >> > >>>>>>> need to upgrade first to 3.4.latest, it is your >> >> decision. >> >> > >>> Definitely >> >> > >>>>> you >> >> > >>>>>>> should test it first, as suggested by the others. >> >> > >>>>>>> >> >> > >>>>>>> Kind regards, >> >> > >>>>>>> Mate >> >> > >>>>>>> >> >> > >>>>>>> On Tue, Mar 24, 2020 at 12:29 PM Norbert Kalmar >> >> > >>>>>>> <nkal...@cloudera.com.invalid> wrote: >> >> > >>>>>>> >> >> > >>>>>>>> Hi, >> >> > >>>>>>>> >> >> > >>>>>>>> That guide is to upgrade to 3.5.0, which was an >> alpha >> >> version. >> >> > >> A >> >> > >>>> lot >> >> > >>>>>> has >> >> > >>>>>>>> changed for the first stable release of 3.5.5 and >> then >> >> a few >> >> > >>> more, >> >> > >>>>> even >> >> > >>>>>>>> rolling upgrade issues have been fixed for 3.5.6. >> >> > >>>>>>>> This is a more up-to-date guide: >> >> > >>>>>>>> >> >> > >>> >> >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ >> >> > >>>>>>>> >> >> > >>>>>>>> If you have done your testing (with prod snapshot!), >> >> then you >> >> > >> can >> >> > >>>>> skip >> >> > >>>>>>> 3.4 >> >> > >>>>>>>> latest upgrade, but keep in mind we do our >> >> recommendations for >> >> > >> a >> >> > >>>>>> reason. >> >> > >>>>>>>> There were issues reported and/or found during >> >> testing. Some >> >> > >> are >> >> > >>>>> fixed >> >> > >>>>>>> with >> >> > >>>>>>>> 3.5.6, some only happens if certain conditions stand >> >> > >>> (IOException: >> >> > >>>> No >> >> > >>>>>>>> snapshot found - mentioned in the guide, fixed in >> >> 3.5.6). >> >> > >>>>>>>> >> >> > >>>>>>>> So it is up to you, I would still recommend to do an >> >> 3.4 >> >> > >> upgrade >> >> > >>>>> first, >> >> > >>>>>>> if >> >> > >>>>>>>> it's feasible. >> >> > >>>>>>>> >> >> > >>>>>>>> Regards, >> >> > >>>>>>>> Norbert >> >> > >>>>>>>> >> >> > >>>>>>>> On Tue, Mar 24, 2020 at 11:45 AM kuldeep singh < >> >> > >>>>>>> kuldeep.sing...@gmail.com> >> >> > >>>>>>>> wrote: >> >> > >>>>>>>> >> >> > >>>>>>>>> Hi, >> >> > >>>>>>>>> >> >> > >>>>>>>>> Current Zookeeper version :- 3.4.5 >> >> > >>>>>>>>> Upgraded version :- 3.5.6 >> >> > >>>>>>>>> >> >> > >>>>>>>>> We are not going with 3.5.7. Our final decision is >> >> zookeeper >> >> > >>>>> version >> >> > >>>>>> is >> >> > >>>>>>>>> 3.5.6 >> >> > >>>>>>>>> as per your reply first we need to move latest >> >> version of >> >> > >>> 3.4.x, >> >> > >>>>> like >> >> > >>>>>>>> below >> >> > >>>>>>>>> >> >> > >>>>>>>>> 3.4.5 -> 3.4.14 -> 3.5.6 (Correct me if I am wrong >> >> here) >> >> > >>>>>>>>> >> >> > >>>>>>>>> But if We are not facing any problem that i have >> >> shared you >> >> > >>> that >> >> > >>>> we >> >> > >>>>>>> have >> >> > >>>>>>>>> set up of 3 node cluster where 2 node are on 3.5.6 >> >> version >> >> > >> and >> >> > >>> 1 >> >> > >>>>> node >> >> > >>>>>>> on >> >> > >>>>>>>>> 3.4.5, Everything is running fine and didn't get >> any >> >> issue, >> >> > >> So >> >> > >>>> what >> >> > >>>>>>> other >> >> > >>>>>>>>> problem we can face if we directly move to 3.5.6 >> >> > >>>>>>>>> >> >> > >>>>>>>>> Thanks, >> >> > >>>>>>>>> --------------------- >> >> > >>>>>>>>> Kuldeep Singh Budania >> >> > >>>>>>>>> Software Architect >> >> > >>>>>>>>> >> >> > >>>>>>>>> >> >> > >>>>>>>>> On Tue, Mar 24, 2020 at 3:58 PM Enrico Olivelli < >> >> > >>>>> eolive...@gmail.com >> >> > >>>>>>> >> >> > >>>>>>>>> wrote: >> >> > >>>>>>>>> >> >> > >>>>>>>>>> Hi >> >> > >>>>>>>>>> You have to upgrade to latest 3.4.x Zookeeper then >> >> you will >> >> > >>>>> upgrade >> >> > >>>>>>> to >> >> > >>>>>>>>>> 3.5.7. >> >> > >>>>>>>>>> All should run well without issues >> >> > >>>>>>>>>> >> >> > >>>>>>>>>> >> >> > >>>>>>>>>> Enrico >> >> > >>>>>>>>>> >> >> > >>>>>>>>>> Il Mar 24 Mar 2020, 10:18 kuldeep singh < >> >> > >>>>> kuldeep.sing...@gmail.com >> >> > >>>>>>> >> >> > >>>>>>> ha >> >> > >>>>>>>>>> scritto: >> >> > >>>>>>>>>> >> >> > >>>>>>>>>>> Hi Team, >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> We are upgrading zookeeper from 3.4.5 to 3.5.6. I >> >> have >> >> > >> set >> >> > >>>> up 3 >> >> > >>>>>>> node >> >> > >>>>>>>>>>> cluster where 2 node are on 3.5.6 version and 1 >> >> node on >> >> > >>>> 3.4.5. >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> Everything is running fine and didn't get any >> issue >> >> on my >> >> > >>>>> system. >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> but I found something on apache site that first >> we >> >> need >> >> > >> to >> >> > >>>>>> upgrade >> >> > >>>>>>>> on >> >> > >>>>>>>>>>> 3.4.6 than we can upgrade to 3.5.6. So is it >> >> mandatory >> >> > >> to >> >> > >>> go >> >> > >>>>> on >> >> > >>>>>>>> 3.4.6 >> >> > >>>>>>>>>>> first. >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> *Upgrading to 3.5.0* >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> Upgrading a running ZooKeeper ensemble to 3.5.0 >> >> should be >> >> > >>>> done >> >> > >>>>>> only >> >> > >>>>>>>>> after >> >> > >>>>>>>>>>> upgrading your ensemble to the 3.4.6 release. >> Note >> >> that >> >> > >>> this >> >> > >>>> is >> >> > >>>>>>> only >> >> > >>>>>>>>>>> necessary for rolling upgrades (if you're fine >> with >> >> > >>> shutting >> >> > >>>>> down >> >> > >>>>>>> the >> >> > >>>>>>>>>>> system completely, you don't have to go through >> >> 3.4.6). >> >> > >> If >> >> > >>>> you >> >> > >>>>>>>> attempt >> >> > >>>>>>>>> a >> >> > >>>>>>>>>>> rolling upgrade without going through 3.4.6 (for >> >> example >> >> > >>> from >> >> > >>>>>>> 3.4.5), >> >> > >>>>>>>>> you >> >> > >>>>>>>>>>> may get the following error: >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - INFO >> [localhost/ >> >> > >>>>>> 127.0.0.1:2784 >> >> > >>>>>>>>>>> :QuorumCnxManager$Listener@498] - Received >> >> connection >> >> > >>>> request >> >> > >>>>> / >> >> > >>>>>>>>>>> 127.0.0.1:60876 >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN >> [localhost/ >> >> > >>>>>> 127.0.0.1:2784 >> >> > >>>>>>>>>>> :QuorumCnxManager@349] - Invalid server id: >> -65536 >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> During a rolling upgrade, each server is taken >> down >> >> in >> >> > >> turn >> >> > >>>> and >> >> > >>>>>>>>> rebooted >> >> > >>>>>>>>>>> with the new 3.5.0 binaries. Before starting the >> >> server >> >> > >>> with >> >> > >>>>>> 3.5.0 >> >> > >>>>>>>>>>> binaries, we highly recommend updating the >> >> configuration >> >> > >>> file >> >> > >>>>> so >> >> > >>>>>>> that >> >> > >>>>>>>>> all >> >> > >>>>>>>>>>> server statements "server.x=..." contain client >> >> ports >> >> > >> (see >> >> > >>>> the >> >> > >>>>>>>> section >> >> > >>>>>>>>>>> Specifying >> >> > >>>>>>>>>>> the client port). As explained earlier you may >> >> leave the >> >> > >>>>>>>> configuration >> >> > >>>>>>>>>> in a >> >> > >>>>>>>>>>> single file, as well as leave the >> >> > >>>> clientPort/clientPortAddress >> >> > >>>>>>>>> statements >> >> > >>>>>>>>>>> (although if you specify client ports in the new >> >> format, >> >> > >>>> these >> >> > >>>>>>>>> statements >> >> > >>>>>>>>>>> are now redundant). >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> Could you please let me know about this case. >> >> Appreciate >> >> > >> if >> >> > >>>>>> respond >> >> > >>>>>>>>> soon. >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>>> Thanks, >> >> > >>>>>>>>>>> --------------------- >> >> > >>>>>>>>>>> Kuldeep Singh Budania >> >> > >>>>>>>>>>> >> >> > >>>>>>>>>> >> >> > >>>>>>>>> >> >> > >>>>>>>> >> >> > >>>>>>> >> >> > >>>>>> >> >> > >>>>> >> >> > >>>> >> >> > >>> >> >> > >> >> >> > >> >> > >> >> >> >> >> >> ________________________________ >> >> >> >> CONFIDENTIALITY & PRIVACY NOTICE >> >> This e-mail (including any attachments) is strictly confidential >> and >> >> may also contain privileged information. If you are not the intended >> >> recipient you are not authorised to read, print, save, process or >> disclose >> >> this message. If you have received this message by mistake, please >> inform >> >> the sender immediately and destroy this e-mail, its attachments and any >> >> copies. Any use, distribution, reproduction or disclosure by any person >> >> other than the intended recipient is strictly prohibited and the person >> >> responsible may incur in penalties. >> >> The use of this e-mail is only for professional purposes; there is >> no >> >> guarantee that the correspondence towards this e-mail will be read >> only by >> >> the recipient, because, under certain circumstances, there may be a >> need to >> >> access this email by third subjects belonging to the Company. >> >> >> >> >> >> ________________________________ >> >> >> >> CONFIDENTIALITY & PRIVACY NOTICE >> >> This e-mail (including any attachments) is strictly confidential and >> may >> >> also contain privileged information. If you are not the intended >> recipient >> >> you are not authorised to read, print, save, process or disclose this >> >> message. If you have received this message by mistake, please inform >> the >> >> sender immediately and destroy this e-mail, its attachments and any >> copies. >> >> Any use, distribution, reproduction or disclosure by any person other >> than >> >> the intended recipient is strictly prohibited and the person >> responsible >> >> may incur in penalties. >> >> The use of this e-mail is only for professional purposes; there is no >> >> guarantee that the correspondence towards this e-mail will be read >> only by >> >> the recipient, because, under certain circumstances, there may be a >> need to >> >> access this email by third subjects belonging to the Company. >> >> >> > >> >
