Thanks for the reply. Now my ZKCli cmd is working fine as we use some our customized authentication and we resolve the issue.
Now I am going to implement Server to Server communication. Thanks, --------------------- Kuldeep Singh Budania On Thu, Jul 2, 2020 at 3:53 PM Szalay-Bekő Máté <szalay.beko.m...@gmail.com> wrote: > I think SSL is working for you already... If you managed to start the > zkCli.sh and be able to connect to ZooKeeper on the secure port and issue > any kind of command (like: " ls / "), then the wire encryption is working > and your server/client communication is secured by ZooKeeper. > > Why you want to run the following command? > addauth ztpasswd zooadmin:<password> > > Do you also want to configure a superDigest user in ZooKeeper? Please note > that this command is independent from SSL. If you need to create a > username-password pair for digest authentication then please use the > command in the following way: > addauth digest zooadmin:yourSuperSecretPassword > > Kind regards, > Mate > > On Thu, Jul 2, 2020 at 6:59 AM kuldeep singh <kuldeep.sing...@gmail.com> > wrote: > > > 1. sh zkCli.sh --config /etc/zookeeper -server localhost:2281 > > > > 2. addauth ztpasswd zooadmin:<password> > > > > > > Thanks, > > --------------------- > > Kuldeep Singh Budania > > > > > > > > On Thu, Jul 2, 2020 at 9:56 AM kuldeep singh <kuldeep.sing...@gmail.com> > > wrote: > > > > > Hi Team, > > > > > > Any update on this? > > > > > > Thanks, > > > --------------------- > > > Kuldeep Singh Budania > > > > > > > > > > > > On Wed, Jul 1, 2020 at 6:43 PM kuldeep singh < > kuldeep.sing...@gmail.com> > > > wrote: > > > > > >> Sorry this is my bad, there were server setting like below > > >> > > >> export SERVER_JVMFLAGS=" > > >> > > >> > > >> > > > -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >> > > >> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > >> -Dzookeeper.ssl.keyStore.password=testpass > > >> > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > >> -Dzookeeper.ssl.trustStore.password=testpass" > > >> > > >> > > >> > > >> export CLIENT_JVMFLAGS=" > > >> > > >> > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > >> -Dzookeeper.client.secure=true > > >> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > >> -Dzookeeper.ssl.keyStore.password=testpass > > >> > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > >> -Dzookeeper.ssl.trustStore.password=testpass" > > >> > > >> I want to have SSL between client to server communication > > >> > > >> I am already following the same link which you have shared with me but > > >> that is not working. > > >> > > >> Zoo.cfg > > >> > > >> secureClientPort=2281 > > >> initLimit=10 > > >> syncLimit=5 > > >> dataDir=/var/lib/zookeeper/data > > >> tickTime=2000 > > >> autopurge.snapRetainCount=3 > > >> autopurge.purgeInterval=1 > > >> admin.enableServer=false > > >> standaloneEnabled=false > > >> jute.maxbuffer=2147483648 > > >> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >> server.1=host1_priv:10288:10388 > > >> server.2=host2_priv:10288:10388 > > >> server.3=host3_priv:10288:10388 > > >> > > >> > > >> command to connect using zkcli > > >> > > >> 1. zkcli zoo.cfg localhost:2281 > > >> 2. addauth ztpasswd usernaem:password > > >> > > >> after second step we are getting below error > > >> > > >> WatchedEvent state:AuthFailed type:None path:null > > >> > > >> > > >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN > > >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication > > provider > > >> for scheme: ztpasswd has x509 ip digest* > > >> > > >> Thanks, > > >> --------------------- > > >> Kuldeep Singh Budania > > >> > > >> > > >> > > >> On Wed, Jul 1, 2020 at 6:25 PM Szalay-Bekő Máté < > > >> szalay.beko.m...@gmail.com> wrote: > > >> > > >>> > No authentication provider for scheme: ztpasswd has x509 ip > digest* > > >>> > > >>> This suggest you have some configuration error... Where did you use > the > > >>> "ztpasswd" string in your configs / commands? > > >>> > > >>> On Wed, Jul 1, 2020 at 2:53 PM Szalay-Bekő Máté < > > >>> szalay.beko.m...@gmail.com> > > >>> wrote: > > >>> > > >>> > > My ZK server is up and running in secure mode > > >>> > > > >>> > What is your goal? You want to setup client-server SSL connection? > > >>> > > > >>> > see: > > >>> > > > >>> > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > > >>> > > > >>> > (or you want to have both SSL and SASL enabled?) > > >>> > > > >>> > Anyway, please remove the following line from the SERVER_JVMFLAGS: > > >>> > > > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > >>> > This is a configuration that makes sense only for the ZooKeeper > > client, > > >>> > not for the server. For the server, use the following: > > >>> > > > >>> > > > >>> > > > -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >>> > > > >>> > If it doesn't solve the issue, then can you please send your > zoo.cfg > > >>> file? > > >>> > Also can you please send the zkCli command you execute? (you need > to > > >>> > connect to the secure ZooKeeper port, unless portUnification is > > >>> enabled) > > >>> > > > >>> > Kind regards, > > >>> > Mate > > >>> > > > >>> > On Wed, Jul 1, 2020 at 9:48 AM kuldeep singh < > > >>> kuldeep.sing...@gmail.com> > > >>> > wrote: > > >>> > > > >>> >> Hi, > > >>> >> > > >>> >> we have done below changes in java.env file > > >>> >> > > >>> >> export SERVER_JVMFLAGS=" > > >>> >> > > >>> >> > > >>> > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > >>> >> > > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > >>> >> -Dzookeeper.ssl.keyStore.password=testpass > > >>> >> > > >>> > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > >>> >> -Dzookeeper.ssl.trustStore.password=testpass" > > >>> >> > > >>> >> > > >>> >> > > >>> >> export CLIENT_JVMFLAGS=" > > >>> >> > > >>> >> > > >>> > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > >>> >> -Dzookeeper.client.secure=true > > >>> >> > > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > >>> >> -Dzookeeper.ssl.keyStore.password=testpass > > >>> >> > > >>> > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > >>> >> -Dzookeeper.ssl.trustStore.password=testpass" > > >>> >> > > >>> >> I have started the ZK server and it is up without any issue. > > >>> >> > > >>> >> But now when I login to ZkCli then it gives the below error. > > >>> >> > > >>> >> WatchedEvent state:AuthFailed type:None path:null > > >>> >> > > >>> >> > > >>> >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN > > >>> >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication > > >>> provider > > >>> >> for scheme: ztpasswd has x509 ip digest* > > >>> >> > > >>> >> Please help me on this issue > > >>> >> > > >>> >> Thanks, > > >>> >> --------------------- > > >>> >> Kuldeep Singh Budania > > >>> >> Software Architect > > >>> >> > > >>> >> > > >>> >> On Wed, Jul 1, 2020 at 12:05 PM kuldeep singh < > > >>> kuldeep.sing...@gmail.com> > > >>> >> wrote: > > >>> >> > > >>> >> > Hi, > > >>> >> > > > >>> >> > My ZK server is up and running in secure mode, But When I am > > >>> trying to > > >>> >> > connect to the ZK server using ZKCli, it gives the below error. > > >>> >> > > > >>> >> > WatchedEvent state:AuthFailed type:None path:null > > >>> >> > > > >>> >> > > > >>> >> > Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN > > >>> >> > [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No > authentication > > >>> >> provider > > >>> >> > for scheme: ztpasswd has x509 ip digest* > > >>> >> > > > >>> >> > Can someone please help me on this issue. we are using the 3.5.6 > > >>> >> version. > > >>> >> > > > >>> >> > I appreciate if I will get a response as soon as possible, as I > am > > >>> stuck > > >>> >> > at this point. > > >>> >> > > > >>> >> > Thanks, > > >>> >> > --------------------- > > >>> >> > Kuldeep Singh Budania > > >>> >> > Software Architect > > >>> >> > > > >>> >> > > > >>> >> > > > >>> >> > On Thu, Jun 25, 2020 at 11:54 AM Enrico Olivelli - Diennea > > >>> >> > <enrico.olive...@diennea.com.invalid> wrote: > > >>> >> > > > >>> >> >> I mean in zoo.cfg > > >>> >> >> Not as a system property > > >>> >> >> > > >>> >> >> Enrico > > >>> >> >> > > >>> >> >> Il giorno 25/06/20, 08:19 "Enrico Olivelli - Diennea" < > > >>> >> >> enrico.olive...@diennea.com.INVALID> ha scritto: > > >>> >> >> > > >>> >> >> Hi > > >>> >> >> You have to enable Netty on the server side > > >>> >> >> > > >>> >> >> Something like: > > >>> >> >> > > >>> >> > > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >>> >> >> > > >>> >> >> Hope that helps > > >>> >> >> Enrico > > >>> >> >> > > >>> >> >> Il giorno 24/06/20, 19:17 "kuldeep singh" < > > >>> >> kuldeep.sing...@gmail.com> > > >>> >> >> ha scritto: > > >>> >> >> > > >>> >> >> Hi, > > >>> >> >> > > >>> >> >> I got below error while setting SSL properties in > > zkEnv.sh > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > ========================================================================================== > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,864 - INFO > > [main:QuorumPeerConfig@133] > > >>> - > > >>> >> >> Reading > > >>> >> >> configuration from: /etc/zookeeper/zoo.cfg > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,874 - INFO > > [main:QuorumPeerConfig@385] > > >>> - > > >>> >> >> clientPortAddress is 0.0.0.0/0.0.0.0:10181 > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,874 - INFO > > [main:QuorumPeerConfig@399] > > >>> - > > >>> >> >> secureClientPortAddress is 0.0.0.0/0.0.0.0:2281 > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,878 - INFO [main:X509Util@79] - > > >>> Setting > > >>> >> -D > > >>> >> >> jdk.tls.rejectClientInitiatedRenegotiation=true to > > disable > > >>> >> >> client-initiated > > >>> >> >> TLS renegotiation > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,897 - INFO > > >>> [main:DatadirCleanupManager@78] > > >>> >> - > > >>> >> >> autopurge.snapRetainCount set to 3 > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,897 - INFO > > >>> [main:DatadirCleanupManager@79] > > >>> >> - > > >>> >> >> autopurge.purgeInterval set to 1 > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,898 - INFO [ > > >>> >> >> PurgeTask:DatadirCleanupManager$PurgeTask@138] - Purge > > >>> task > > >>> >> >> started. > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,899 - INFO [main:ManagedUtil@46] > - > > >>> Log4j > > >>> >> >> found with > > >>> >> >> jmx enabled. > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,903 - INFO > > >>> [PurgeTask:FileTxnSnapLog@103] > > >>> >> - > > >>> >> >> zookeeper.snapshot.trust.empty : false > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,910 - INFO [ > > >>> >> >> PurgeTask:DatadirCleanupManager$PurgeTask@144] - Purge > > >>> task > > >>> >> >> completed. > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,975 - INFO > [main:QuorumPeerMain@141 > > ] > > >>> - > > >>> >> >> Starting quorum > > >>> >> >> peer > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,983 - INFO > > [main:ServerCnxnFactory@135] > > >>> - > > >>> >> >> Using > > >>> >> >> org.apache.zookeeper.server.NIOServerCnxnFactory as > > server > > >>> >> >> connection > > >>> >> >> factory > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,986 - INFO > > >>> [main:NIOServerCnxnFactory@673] > > >>> >> - > > >>> >> >> Configuring NIO connection handler with 10s sessionless > > >>> >> >> connection timeout, > > >>> >> >> 2 selector thread(s), 16 worker threads, and 64 kB > direct > > >>> >> buffers. > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,992 - INFO > > >>> [main:NIOServerCnxnFactory@686 > > >>> >> ] > > >>> >> >> - binding > > >>> >> >> to port 0.0.0.0/0.0.0.0:10181 > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,994 - INFO > > [main:ServerCnxnFactory@135] > > >>> - > > >>> >> >> Using > > >>> >> >> org.apache.zookeeper.server.NIOServerCnxnFactory as > > server > > >>> >> >> connection > > >>> >> >> factory > > >>> >> >> > > >>> >> >> 2020-06-24 15:49:35,995 - ERROR > [main:QuorumPeerMain@101 > > ] > > >>> - > > >>> >> >> Unexpected > > >>> >> >> exception, exiting abnormally > > >>> >> >> > > >>> >> >> java.lang.UnsupportedOperationException: SSL isn't > > >>> supported in > > >>> >> >> NIOServerCnxn > > >>> >> >> > > >>> >> >> at > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:644) > > >>> >> >> > > >>> >> >> at > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:155) > > >>> >> >> > > >>> >> >> at > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) > > >>> >> >> > > >>> >> >> at > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> I have set the following properties in SERVER_JVMFLAGS > in > > >>> >> >> zkEnv.sh file : > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > "-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > -Dzookeeper.ssl.keyStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkkeystore.p12 > > >>> >> >> -Dzookeeper.ssl.keyStore.password=EvaiKiO1@123456 > > >>> >> >> > > >>> >> >> > > >>> >> > > >>> > > > -Dzookeeper.ssl.trustStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkTrustStore.jks > > >>> >> >> -Dzookeeper.ssl.trustStore.password=EvaiKiO1@123456" > > >>> >> >> > > >>> >> >> Thanks, > > >>> >> >> --------------------- > > >>> >> >> Kuldeep Singh Budania > > >>> >> >> > > >>> >> >> > > >>> >> >> > > >>> >> >> On Mon, Jun 22, 2020 at 8:08 PM Jordan Zimmerman < > > >>> >> >> jor...@jordanzimmerman.com> > > >>> >> >> wrote: > > >>> >> >> > > >>> >> >> > It's the same as the normal ZooKeeper client: > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions > > >>> >> >> > < > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions > > >>> >> >> > > > > >>> >> >> > > > >>> >> >> > -Jordan > > >>> >> >> > > > >>> >> >> > > On Jun 22, 2020, at 5:50 AM, kuldeep singh < > > >>> >> >> kuldeep.sing...@gmail.com> > > >>> >> >> > wrote: > > >>> >> >> > > > > >>> >> >> > > Hi Team, > > >>> >> >> > > How we will do secure communication between the > > Curator > > >>> >> >> framework and > > >>> >> >> > > zookeeper 3.5.6 ? > > >>> >> >> > > I didn't get any solution right now. > > >>> >> >> > > I appreciate it if someone could help me with the > > same. > > >>> >> >> > > > > >>> >> >> > > Thanks, > > >>> >> >> > > --------------------- > > >>> >> >> > > Kuldeep Singh Budania > > >>> >> >> > > Software Architect > > >>> >> >> > > > > >>> >> >> > > > > >>> >> >> > > On Fri, Apr 17, 2020 at 4:53 PM Szalay-Bekő Máté < > > >>> >> >> > szalay.beko.m...@gmail.com> > > >>> >> >> > > wrote: > > >>> >> >> > > > > >>> >> >> > >> Hello Kuldeep, > > >>> >> >> > >> > > >>> >> >> > >> did you download the source from the ZooKeeper > > >>> webpage, or > > >>> >> >> checked out > > >>> >> >> > from > > >>> >> >> > >> git? > > >>> >> >> > >> Anyway, the following commands should work: > > >>> >> >> > >> > > >>> >> >> > >> wget > > >>> >> >> > >> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > https://downloads.apache.org/zookeeper/zookeeper-3.5.7/apache-zookeeper-3.5.7.tar.gz > > >>> >> >> > >> tar xzvf ./apache-zookeeper-3.5.7.tar.gz > > >>> >> >> > >> cd apache-zookeeper-3.5.7 > > >>> >> >> > >> mvn clean install -DskipTests > > >>> >> >> > >> > > >>> >> >> > >> I tested it with OpenJDK 8u424 and maven 3.6.0. > > >>> >> >> > >> > > >>> >> >> > >> Kind regards, > > >>> >> >> > >> Mate > > >>> >> >> > >> > > >>> >> >> > >> > > >>> >> >> > >> On Fri, Apr 17, 2020 at 12:51 PM kuldeep singh < > > >>> >> >> > kuldeep.sing...@gmail.com> > > >>> >> >> > >> wrote: > > >>> >> >> > >> > > >>> >> >> > >>> Hi All, > > >>> >> >> > >>> > > >>> >> >> > >>> I have download the zookeeper source code and run > > >>> the mvn > > >>> >> >> command for > > >>> >> >> > >>> packaging but get problem on "zookeeper-server" > > >>> >> directory. > > >>> >> >> > >>> > > >>> >> >> > >>> *command :- mvn clean package* > > >>> >> >> > >>> > > >>> >> >> > >>> [ERROR] Failed to execute goal > > >>> >> >> > >>> > > >>> >> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile > > >>> >> >> > >>> (default-compile) on project zookeeper: Fatal > error > > >>> >> >> compiling: > > >>> >> >> > >>> java.lang.NullPointerException -> [Help 1] > > >>> >> >> > >>> > > >>> org.apache.maven.lifecycle.LifecycleExecutionException: > > >>> >> >> Failed to > > >>> >> >> > execute > > >>> >> >> > >>> goal > > >>> >> >> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile > > >>> >> >> > >>> (default-compile) on project zookeeper: Fatal > error > > >>> >> >> compiling > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) > > >>> >> >> > >>> at > > >>> >> >> > > > >>> >> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) > > >>> >> >> > >>> at > > >>> >> >> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) > > >>> >> >> > >>> at > > >>> >> >> org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) > > >>> >> >> > >>> at > > >>> >> >> org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) > > >>> >> >> > >>> at > > >>> >> >> org.apache.maven.cli.MavenCli.main(MavenCli.java:157) > > >>> >> >> > >>> at > > >>> >> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > >>> >> >> > >>> at > > >>> >> java.lang.reflect.Method.invoke(Method.java:498) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > > >>> >> >> > >>> Caused by: > > >>> >> org.apache.maven.plugin.MojoExecutionException: > > >>> >> >> Fatal error > > >>> >> >> > >>> compiling > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1140) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:188) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) > > >>> >> >> > >>> ... 19 more > > >>> >> >> > >>> Caused by: > > >>> >> org.codehaus.plexus.compiler.CompilerException: > > >>> >> >> > >>> java.lang.NullPointerException > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:173) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1129) > > >>> >> >> > >>> ... 22 more > > >>> >> >> > >>> Caused by: java.lang.RuntimeException: > > >>> >> >> java.lang.NullPointerException > > >>> >> >> > >>> at > > >>> >> >> com.sun.tools.javac.main.Main.compile(Main.java:559) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126) > > >>> >> >> > >>> ... 24 more > > >>> >> >> > >>> Caused by: java.lang.NullPointerException > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > com.sun.tools.javac.model.JavacTypes.getOverriddenMethods(JavacTypes.java:306) > > >>> >> >> > >>> at > > >>> >> com.sun.tools.doclint.Env.setCurrent(Env.java:155) > > >>> >> >> > >>> at > > >>> >> >> com.sun.tools.doclint.Checker.scan(Checker.java:144) > > >>> >> >> > >>> at > > >>> >> >> com.sun.tools.doclint.DocLint$2.visitDecl(DocLint.java:285) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:359) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.visitNewClass(TreeScanner.java:280) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.tools.javac.tree.JCTree$JCNewClass.accept(JCTree.java:1532) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.visitVariable(TreeScanner.java:153) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:373) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:346) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> > com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:864) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > >>> >> >> > >>> at > > >>> >> >> com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) > > >>> >> >> > >>> at > > >>> >> >> > >> > > >>> >> >> > com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:133) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:360) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> >> >> > >>> at > > >>> >> >> com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:120) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:354) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:346) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:550) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > >>> >> >> > >>> at > > >>> >> >> com.sun.tools.doclint.DocLint$3.started(DocLint.java:296) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.started(ClientCodeWrapper.java:668) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > com.sun.tools.javac.api.MultiTaskListener.started(MultiTaskListener.java:103) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> > com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1240) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901) > > >>> >> >> > >>> at > > >>> >> >> > >>> > > >>> >> >> > > >>> com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860) > > >>> >> >> > >>> at > > >>> >> >> com.sun.tools.javac.main.Main.compile(Main.java:523) > > >>> >> >> > >>> ... 27 more > > >>> >> >> > >>> [ERROR] > > >>> >> >> > >>> [ERROR] Re-run Maven using the -X switch to > enable > > >>> full > > >>> >> >> debug logging. > > >>> >> >> > >>> [ERROR] > > >>> >> >> > >>> [ERROR] For more information about the errors and > > >>> >> possible > > >>> >> >> solutions, > > >>> >> >> > >>> please read the following articles: > > >>> >> >> > >>> [ERROR] [Help 1] > > >>> >> >> > >>> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > > >>> >> >> > >>> > > >>> >> >> > >>> Can someone help me on this issue? > > >>> >> >> > >>> > > >>> >> >> > >>> Thanks, > > >>> >> >> > >>> --------------------- > > >>> >> >> > >>> Kuldeep Singh Budania > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >>> > > >>> >> >> > >>> On Sat, Apr 4, 2020 at 5:57 PM Szalay-Bekő Máté < > > >>> >> >> > >>> szalay.beko.m...@gmail.com> > > >>> >> >> > >>> wrote: > > >>> >> >> > >>> > > >>> >> >> > >>>> these exceptions can mean many things... I think > > >>> this > > >>> >> can > > >>> >> >> be even > > >>> >> >> > >> normal > > >>> >> >> > >>>> duding rolling restart (as some connections get > > >>> broken > > >>> >> in > > >>> >> >> this case) > > >>> >> >> > >>>> > > >>> >> >> > >>>> However, I saw cases already when exceptions > like > > >>> these > > >>> >> >> killed > > >>> >> >> > receiver > > >>> >> >> > >>> or > > >>> >> >> > >>>> sender threads in QuorumCnxManager / Leader > > >>> Election in > > >>> >> >> such a way > > >>> >> >> > that > > >>> >> >> > >>>> they were not able to recover, so the node was > > >>> unable to > > >>> >> >> connect to > > >>> >> >> > any > > >>> >> >> > >>>> quorum until restart. I remember seeing this in > > 3.4 > > >>> too. > > >>> >> >> > >>>> > > >>> >> >> > >>>> Do you see these exceptions in the second server > > >>> (the > > >>> >> one > > >>> >> >> which you > > >>> >> >> > >> just > > >>> >> >> > >>>> upgraded in step 3)? > > >>> >> >> > >>>> Is this issue reproducible? > > >>> >> >> > >>>> > > >>> >> >> > >>>> What is the tickTime and initLimit you use? > Maybe > > >>> the > > >>> >> >> server just > > >>> >> >> > >> require > > >>> >> >> > >>>> more time to sync? > > >>> >> >> > >>>> > > >>> >> >> > >>>> I would need more logs to really see what > > happened. > > >>> Can > > >>> >> >> you create a > > >>> >> >> > >> Jira > > >>> >> >> > >>>> issue and upload the logs and also the ZooKeeper > > >>> >> configs? > > >>> >> >> I am happy > > >>> >> >> > to > > >>> >> >> > >>>> take a closer look. > > >>> >> >> > >>>> (if you need to re-run the test to collect the > > logs, > > >>> >> then > > >>> >> >> enabling > > >>> >> >> > >> DEBUG > > >>> >> >> > >>>> logs would be great. The INFO level logs are > > usually > > >>> >> >> enough for these > > >>> >> >> > >>>> problems, but one can never know...) > > >>> >> >> > >>>> > > >>> >> >> > >>>> Kind regards, > > >>> >> >> > >>>> Mate > > >>> >> >> > >>>> > > >>> >> >> > >>>> > > >>> >> >> > >>>> On Fri, Apr 3, 2020 at 10:05 AM kuldeep singh < > > >>> >> >> > >> kuldeep.sing...@gmail.com > > >>> >> >> > >>>> > > >>> >> >> > >>>> wrote: > > >>> >> >> > >>>> > > >>> >> >> > >>>>> Hi Team, > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> I have done some POC on rolling upgrade and > found > > >>> below > > >>> >> >> result. > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> 1. On 1st node upgrade zookeeper . Traffic > was > > >>> >> running > > >>> >> >> fine > > >>> >> >> > >> because > > >>> >> >> > >>> 2 > > >>> >> >> > >>>>> nodes are already on old zookeeper. > > >>> >> >> > >>>>> 2. On 1st node upgrade our application and > > didn’t > > >>> >> find > > >>> >> >> any issue > > >>> >> >> > >>>>> 3. On 2nd node upgrade zookeeper but got > below > > >>> error > > >>> >> >> and zookeeper > > >>> >> >> > >>> is > > >>> >> >> > >>>>> not taking any requests > > >>> >> >> > >>>>> 4. > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> java.io.EOFException > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> java.io.DataInputStream.readInt(DataInputStream.java:392) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:747) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> 2020-03-30 14:19:55,587 - WARN > > >>> >> >> > >>>>> [RecvWorker:1:QuorumCnxManager$RecvWorker@765] > - > > >>> >> >> Interrupting > > >>> >> >> > >>> SendWorker > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> 2020-03-30 14:19:55,588 - ERROR > [LearnerHandler-/ > > >>> >> >> 192.168.44.73:33754 > > >>> >> >> > >>>>> :LearnerHandler@562] - Unexpected exception > > >>> causing > > >>> >> >> shutdown while > > >>> >> >> > >>> sock > > >>> >> >> > >>>>> still open > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> java.io.EOFException > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> java.io.DataInputStream.readInt(DataInputStream.java:392) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> > >>>>> > > >>> >> >> > >> > > >>> >> >> > > >>> > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> > >>>>> > > >>> >> >> > >>>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> at > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > >>> >> > > >>> > > > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:476) > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> 2020-03-30 14:19:55,588 - WARN > > >>> >> >> > >>>>> [SendWorker:1:QuorumCnxManager$SendWorker@679] > - > > >>> >> >> Interrupted while > > >>> >> >> > >>>> waiting > > >>> >> >> > >>>>> for message on queue > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> Please let me know is this the known issue or > > this > > >>> is > > >>> >> >> different issue > > >>> >> >> > >>>> which > > >>> >> >> > >>>>> is mention in Apache zookeeper documentation > when > > >>> >> >> upgrading from > > >>> >> >> > >> 3.4.5 > > >>> >> >> > >>> to > > >>> >> >> > >>>>> 3.5.6 > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> Thanks, > > >>> >> >> > >>>>> --------------------- > > >>> >> >> > >>>>> Kuldeep Singh Budania > > >>> >> >> > >>>>> Software Architect > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>>> On Sun, Mar 29, 2020 at 9:06 AM Alexander > Shraer > > < > > >>> >> >> shra...@gmail.com> > > >>> >> >> > >>>>> wrote: > > >>> >> >> > >>>>> > > >>> >> >> > >>>>>> +1 to what Mate said (I wrote the quoted > > >>> >> instructions). > > >>> >> >> > >>>>>> > > >>> >> >> > >>>>>> > > >>> >> >> > >>>>>> > > >>> >> >> > >>>>>> On Tue, Mar 24, 2020 at 7:03 AM Szalay-Bekő > > Máté < > > >>> >> >> > >>>>>> szalay.beko.m...@gmail.com> > > >>> >> >> > >>>>>> wrote: > > >>> >> >> > >>>>>> > > >>> >> >> > >>>>>>> Hi Kuldeep, > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> I just want to provide you some background > info > > >>> about > > >>> >> >> our > > >>> >> >> > >>>>> documentation. > > >>> >> >> > >>>>>>> The reason to upgrade to 3.4.6 first is to > > avoid > > >>> the > > >>> >> >> following > > >>> >> >> > >>> error: > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN > > >>> [localhost/ > > >>> >> >> > >>> 127.0.0.1:2784 > > >>> >> >> > >>>>>>> :QuorumCnxManager@349] - Invalid server id: > > >>> -65536 > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> This error comes because of the protocol > > changes > > >>> >> between > > >>> >> >> > >> ZooKeeper > > >>> >> >> > >>>>> server > > >>> >> >> > >>>>>>> nodes during connection initiation for leader > > >>> >> election. > > >>> >> >> In > > >>> >> >> > >>> ZooKeeper > > >>> >> >> > >>>>> 3.5 > > >>> >> >> > >>>>>> a > > >>> >> >> > >>>>>>> protocol version was introduced (see > > >>> ZOOKEEPER-107) > > >>> >> and > > >>> >> >> since > > >>> >> >> > >> that > > >>> >> >> > >>>> time > > >>> >> >> > >>>>>> the > > >>> >> >> > >>>>>>> fist long value sent in the initial message > is > > >>> not > > >>> >> the > > >>> >> >> server ID > > >>> >> >> > >>> but > > >>> >> >> > >>>>> the > > >>> >> >> > >>>>>>> protocol version (-65536). In ZooKeeper 3.4.6 > > we > > >>> made > > >>> >> >> the old 3.4 > > >>> >> >> > >>>>>>> ZooKeepers backward compatible, so they are > > able > > >>> to > > >>> >> >> parse both > > >>> >> >> > >> the > > >>> >> >> > >>>> old > > >>> >> >> > >>>>>> and > > >>> >> >> > >>>>>>> the new protocol format (see ZOOKEEPER-1633). > > >>> This > > >>> >> >> issue happens > > >>> >> >> > >>> only > > >>> >> >> > >>>>>> when > > >>> >> >> > >>>>>>> you need to use old (3.4.0 - 3.4.5) and new > > >>> (3.5.0+) > > >>> >> >> ZooKeeper > > >>> >> >> > >>>> servers > > >>> >> >> > >>>>>>> together in the same cluster. During a > rolling > > >>> >> upgrade, > > >>> >> >> this is > > >>> >> >> > >>>> usually > > >>> >> >> > >>>>>> the > > >>> >> >> > >>>>>>> case to have old and new ZooKeepers present > > >>> together. > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> The fact that you haven't seen any issues > might > > >>> be > > >>> >> >> caused by the > > >>> >> >> > >>>> order > > >>> >> >> > >>>>> of > > >>> >> >> > >>>>>>> the servers. In ZooKeeper the connection > > >>> initiation > > >>> >> >> between the > > >>> >> >> > >>>> servers > > >>> >> >> > >>>>>>> during the leader election follows a specific > > >>> rule. > > >>> >> As > > >>> >> >> far as I > > >>> >> >> > >>>>> remember > > >>> >> >> > >>>>>>> always the server with the larger ID 'wins > the > > >>> >> >> challenge', so it > > >>> >> >> > >> is > > >>> >> >> > >>>>>>> possible, that the old server didn't need to > > >>> parse > > >>> >> any > > >>> >> >> initial > > >>> >> >> > >>>> message > > >>> >> >> > >>>>>> (if > > >>> >> >> > >>>>>>> it had the largest ID) and this is why you > > >>> haven't > > >>> >> seen > > >>> >> >> the > > >>> >> >> > >> issue. > > >>> >> >> > >>>> Also > > >>> >> >> > >>>>>>> having 2 nodes up from the 3 nodes cluster > > still > > >>> >> makes > > >>> >> >> the > > >>> >> >> > >> cluster > > >>> >> >> > >>>> work > > >>> >> >> > >>>>>> (so > > >>> >> >> > >>>>>>> you should also check if all the servers are > > >>> part of > > >>> >> >> the quorum). > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> I agree with Enrico and Norbert, the safest > and > > >>> most > > >>> >> >> stable way > > >>> >> >> > >> is > > >>> >> >> > >>>>>> upgrade > > >>> >> >> > >>>>>>> first to 3.4.latest, then go to 3.5.latest. > > >>> Still, if > > >>> >> >> you don't > > >>> >> >> > >> see > > >>> >> >> > >>>>> that > > >>> >> >> > >>>>>>> you would hit this specific issue (e.g. no > > >>> "Invalid > > >>> >> >> server id" in > > >>> >> >> > >>> the > > >>> >> >> > >>>>> log > > >>> >> >> > >>>>>>> files), and all the three servers can handle > > >>> traffic, > > >>> >> >> then maybe > > >>> >> >> > >>> you > > >>> >> >> > >>>>>> don't > > >>> >> >> > >>>>>>> need to upgrade first to 3.4.latest, it is > your > > >>> >> >> decision. > > >>> >> >> > >>> Definitely > > >>> >> >> > >>>>> you > > >>> >> >> > >>>>>>> should test it first, as suggested by the > > others. > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> Kind regards, > > >>> >> >> > >>>>>>> Mate > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> On Tue, Mar 24, 2020 at 12:29 PM Norbert > Kalmar > > >>> >> >> > >>>>>>> <nkal...@cloudera.com.invalid> wrote: > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>>> Hi, > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>>> That guide is to upgrade to 3.5.0, which was > > an > > >>> >> alpha > > >>> >> >> version. > > >>> >> >> > >> A > > >>> >> >> > >>>> lot > > >>> >> >> > >>>>>> has > > >>> >> >> > >>>>>>>> changed for the first stable release of > 3.5.5 > > >>> and > > >>> >> then > > >>> >> >> a few > > >>> >> >> > >>> more, > > >>> >> >> > >>>>> even > > >>> >> >> > >>>>>>>> rolling upgrade issues have been fixed for > > >>> 3.5.6. > > >>> >> >> > >>>>>>>> This is a more up-to-date guide: > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>> > > >>> >> >> > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>>> If you have done your testing (with prod > > >>> snapshot!), > > >>> >> >> then you > > >>> >> >> > >> can > > >>> >> >> > >>>>> skip > > >>> >> >> > >>>>>>> 3.4 > > >>> >> >> > >>>>>>>> latest upgrade, but keep in mind we do our > > >>> >> >> recommendations for > > >>> >> >> > >> a > > >>> >> >> > >>>>>> reason. > > >>> >> >> > >>>>>>>> There were issues reported and/or found > during > > >>> >> >> testing. Some > > >>> >> >> > >> are > > >>> >> >> > >>>>> fixed > > >>> >> >> > >>>>>>> with > > >>> >> >> > >>>>>>>> 3.5.6, some only happens if certain > conditions > > >>> stand > > >>> >> >> > >>> (IOException: > > >>> >> >> > >>>> No > > >>> >> >> > >>>>>>>> snapshot found - mentioned in the guide, > fixed > > >>> in > > >>> >> >> 3.5.6). > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>>> So it is up to you, I would still recommend > to > > >>> do an > > >>> >> >> 3.4 > > >>> >> >> > >> upgrade > > >>> >> >> > >>>>> first, > > >>> >> >> > >>>>>>> if > > >>> >> >> > >>>>>>>> it's feasible. > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>>> Regards, > > >>> >> >> > >>>>>>>> Norbert > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>>> On Tue, Mar 24, 2020 at 11:45 AM kuldeep > > singh < > > >>> >> >> > >>>>>>> kuldeep.sing...@gmail.com> > > >>> >> >> > >>>>>>>> wrote: > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>>>> Hi, > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> Current Zookeeper version :- 3.4.5 > > >>> >> >> > >>>>>>>>> Upgraded version :- 3.5.6 > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> We are not going with 3.5.7. Our final > > >>> decision is > > >>> >> >> zookeeper > > >>> >> >> > >>>>> version > > >>> >> >> > >>>>>> is > > >>> >> >> > >>>>>>>>> 3.5.6 > > >>> >> >> > >>>>>>>>> as per your reply first we need to move > > latest > > >>> >> >> version of > > >>> >> >> > >>> 3.4.x, > > >>> >> >> > >>>>> like > > >>> >> >> > >>>>>>>> below > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> 3.4.5 -> 3.4.14 -> 3.5.6 (Correct me if I > am > > >>> wrong > > >>> >> >> here) > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> But if We are not facing any problem that i > > >>> have > > >>> >> >> shared you > > >>> >> >> > >>> that > > >>> >> >> > >>>> we > > >>> >> >> > >>>>>>> have > > >>> >> >> > >>>>>>>>> set up of 3 node cluster where 2 node are > on > > >>> 3.5.6 > > >>> >> >> version > > >>> >> >> > >> and > > >>> >> >> > >>> 1 > > >>> >> >> > >>>>> node > > >>> >> >> > >>>>>>> on > > >>> >> >> > >>>>>>>>> 3.4.5, Everything is running fine and > didn't > > >>> get > > >>> >> any > > >>> >> >> issue, > > >>> >> >> > >> So > > >>> >> >> > >>>> what > > >>> >> >> > >>>>>>> other > > >>> >> >> > >>>>>>>>> problem we can face if we directly move to > > >>> 3.5.6 > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> Thanks, > > >>> >> >> > >>>>>>>>> --------------------- > > >>> >> >> > >>>>>>>>> Kuldeep Singh Budania > > >>> >> >> > >>>>>>>>> Software Architect > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>> On Tue, Mar 24, 2020 at 3:58 PM Enrico > > >>> Olivelli < > > >>> >> >> > >>>>> eolive...@gmail.com > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>>>> wrote: > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>>>> Hi > > >>> >> >> > >>>>>>>>>> You have to upgrade to latest 3.4.x > > Zookeeper > > >>> then > > >>> >> >> you will > > >>> >> >> > >>>>> upgrade > > >>> >> >> > >>>>>>> to > > >>> >> >> > >>>>>>>>>> 3.5.7. > > >>> >> >> > >>>>>>>>>> All should run well without issues > > >>> >> >> > >>>>>>>>>> > > >>> >> >> > >>>>>>>>>> > > >>> >> >> > >>>>>>>>>> Enrico > > >>> >> >> > >>>>>>>>>> > > >>> >> >> > >>>>>>>>>> Il Mar 24 Mar 2020, 10:18 kuldeep singh < > > >>> >> >> > >>>>> kuldeep.sing...@gmail.com > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>>> ha > > >>> >> >> > >>>>>>>>>> scritto: > > >>> >> >> > >>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> Hi Team, > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> We are upgrading zookeeper from 3.4.5 to > > >>> 3.5.6. I > > >>> >> >> have > > >>> >> >> > >> set > > >>> >> >> > >>>> up 3 > > >>> >> >> > >>>>>>> node > > >>> >> >> > >>>>>>>>>>> cluster where 2 node are on 3.5.6 version > > >>> and 1 > > >>> >> >> node on > > >>> >> >> > >>>> 3.4.5. > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> Everything is running fine and didn't get > > any > > >>> >> issue > > >>> >> >> on my > > >>> >> >> > >>>>> system. > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> but I found something on apache site > that > > >>> first > > >>> >> we > > >>> >> >> need > > >>> >> >> > >> to > > >>> >> >> > >>>>>> upgrade > > >>> >> >> > >>>>>>>> on > > >>> >> >> > >>>>>>>>>>> 3.4.6 than we can upgrade to 3.5.6. So is > > it > > >>> >> >> mandatory > > >>> >> >> > >> to > > >>> >> >> > >>> go > > >>> >> >> > >>>>> on > > >>> >> >> > >>>>>>>> 3.4.6 > > >>> >> >> > >>>>>>>>>>> first. > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> *Upgrading to 3.5.0* > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> Upgrading a running ZooKeeper ensemble to > > >>> 3.5.0 > > >>> >> >> should be > > >>> >> >> > >>>> done > > >>> >> >> > >>>>>> only > > >>> >> >> > >>>>>>>>> after > > >>> >> >> > >>>>>>>>>>> upgrading your ensemble to the 3.4.6 > > release. > > >>> >> Note > > >>> >> >> that > > >>> >> >> > >>> this > > >>> >> >> > >>>> is > > >>> >> >> > >>>>>>> only > > >>> >> >> > >>>>>>>>>>> necessary for rolling upgrades (if you're > > >>> fine > > >>> >> with > > >>> >> >> > >>> shutting > > >>> >> >> > >>>>> down > > >>> >> >> > >>>>>>> the > > >>> >> >> > >>>>>>>>>>> system completely, you don't have to go > > >>> through > > >>> >> >> 3.4.6). > > >>> >> >> > >> If > > >>> >> >> > >>>> you > > >>> >> >> > >>>>>>>> attempt > > >>> >> >> > >>>>>>>>> a > > >>> >> >> > >>>>>>>>>>> rolling upgrade without going through > 3.4.6 > > >>> (for > > >>> >> >> example > > >>> >> >> > >>> from > > >>> >> >> > >>>>>>> 3.4.5), > > >>> >> >> > >>>>>>>>> you > > >>> >> >> > >>>>>>>>>>> may get the following error: > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - INFO > > >>> >> [localhost/ > > >>> >> >> > >>>>>> 127.0.0.1:2784 > > >>> >> >> > >>>>>>>>>>> :QuorumCnxManager$Listener@498] - > Received > > >>> >> >> connection > > >>> >> >> > >>>> request > > >>> >> >> > >>>>> / > > >>> >> >> > >>>>>>>>>>> 127.0.0.1:60876 > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN > > >>> >> [localhost/ > > >>> >> >> > >>>>>> 127.0.0.1:2784 > > >>> >> >> > >>>>>>>>>>> :QuorumCnxManager@349] - Invalid server > > id: > > >>> >> -65536 > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> During a rolling upgrade, each server is > > >>> taken > > >>> >> down > > >>> >> >> in > > >>> >> >> > >> turn > > >>> >> >> > >>>> and > > >>> >> >> > >>>>>>>>> rebooted > > >>> >> >> > >>>>>>>>>>> with the new 3.5.0 binaries. Before > > starting > > >>> the > > >>> >> >> server > > >>> >> >> > >>> with > > >>> >> >> > >>>>>> 3.5.0 > > >>> >> >> > >>>>>>>>>>> binaries, we highly recommend updating > the > > >>> >> >> configuration > > >>> >> >> > >>> file > > >>> >> >> > >>>>> so > > >>> >> >> > >>>>>>> that > > >>> >> >> > >>>>>>>>> all > > >>> >> >> > >>>>>>>>>>> server statements "server.x=..." contain > > >>> client > > >>> >> >> ports > > >>> >> >> > >> (see > > >>> >> >> > >>>> the > > >>> >> >> > >>>>>>>> section > > >>> >> >> > >>>>>>>>>>> Specifying > > >>> >> >> > >>>>>>>>>>> the client port). As explained earlier > you > > >>> may > > >>> >> >> leave the > > >>> >> >> > >>>>>>>> configuration > > >>> >> >> > >>>>>>>>>> in a > > >>> >> >> > >>>>>>>>>>> single file, as well as leave the > > >>> >> >> > >>>> clientPort/clientPortAddress > > >>> >> >> > >>>>>>>>> statements > > >>> >> >> > >>>>>>>>>>> (although if you specify client ports in > > the > > >>> new > > >>> >> >> format, > > >>> >> >> > >>>> these > > >>> >> >> > >>>>>>>>> statements > > >>> >> >> > >>>>>>>>>>> are now redundant). > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> Could you please let me know about this > > case. > > >>> >> >> Appreciate > > >>> >> >> > >> if > > >>> >> >> > >>>>>> respond > > >>> >> >> > >>>>>>>>> soon. > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>>> Thanks, > > >>> >> >> > >>>>>>>>>>> --------------------- > > >>> >> >> > >>>>>>>>>>> Kuldeep Singh Budania > > >>> >> >> > >>>>>>>>>>> > > >>> >> >> > >>>>>>>>>> > > >>> >> >> > >>>>>>>>> > > >>> >> >> > >>>>>>>> > > >>> >> >> > >>>>>>> > > >>> >> >> > >>>>>> > > >>> >> >> > >>>>> > > >>> >> >> > >>>> > > >>> >> >> > >>> > > >>> >> >> > >> > > >>> >> >> > > > >>> >> >> > > > >>> >> >> > > >>> >> >> > > >>> >> >> ________________________________ > > >>> >> >> > > >>> >> >> CONFIDENTIALITY & PRIVACY NOTICE > > >>> >> >> This e-mail (including any attachments) is strictly > > >>> confidential > > >>> >> and > > >>> >> >> may also contain privileged information. If you are not the > > >>> intended > > >>> >> >> recipient you are not authorised to read, print, save, process > or > > >>> >> disclose > > >>> >> >> this message. If you have received this message by mistake, > > please > > >>> >> inform > > >>> >> >> the sender immediately and destroy this e-mail, its attachments > > >>> and any > > >>> >> >> copies. Any use, distribution, reproduction or disclosure by > any > > >>> person > > >>> >> >> other than the intended recipient is strictly prohibited and > the > > >>> person > > >>> >> >> responsible may incur in penalties. > > >>> >> >> The use of this e-mail is only for professional purposes; > > >>> there is > > >>> >> no > > >>> >> >> guarantee that the correspondence towards this e-mail will be > > read > > >>> >> only by > > >>> >> >> the recipient, because, under certain circumstances, there may > > be a > > >>> >> need to > > >>> >> >> access this email by third subjects belonging to the Company. > > >>> >> >> > > >>> >> >> > > >>> >> >> ________________________________ > > >>> >> >> > > >>> >> >> CONFIDENTIALITY & PRIVACY NOTICE > > >>> >> >> This e-mail (including any attachments) is strictly > confidential > > >>> and > > >>> >> may > > >>> >> >> also contain privileged information. If you are not the > intended > > >>> >> recipient > > >>> >> >> you are not authorised to read, print, save, process or > disclose > > >>> this > > >>> >> >> message. If you have received this message by mistake, please > > >>> inform > > >>> >> the > > >>> >> >> sender immediately and destroy this e-mail, its attachments and > > any > > >>> >> copies. > > >>> >> >> Any use, distribution, reproduction or disclosure by any person > > >>> other > > >>> >> than > > >>> >> >> the intended recipient is strictly prohibited and the person > > >>> >> responsible > > >>> >> >> may incur in penalties. > > >>> >> >> The use of this e-mail is only for professional purposes; there > > is > > >>> no > > >>> >> >> guarantee that the correspondence towards this e-mail will be > > read > > >>> >> only by > > >>> >> >> the recipient, because, under certain circumstances, there may > > be a > > >>> >> need to > > >>> >> >> access this email by third subjects belonging to the Company. > > >>> >> >> > > >>> >> > > > >>> >> > > >>> > > > >>> > > >> > > >
