Hi Team, Server to Server communication is not supported in 3.5.6 version as per below JIRA issue?
https://issues.apache.org/jira/browse/ZOOKEEPER-2639 Thanks, --------------------- Kuldeep Singh Budania On Thu, Jul 2, 2020 at 4:24 PM kuldeep singh <kuldeep.sing...@gmail.com> wrote: > Thanks for the reply. > > Now my ZKCli cmd is working fine as we use some our customized > authentication and we resolve the issue. > > Now I am going to implement Server to Server communication. > > Thanks, > --------------------- > Kuldeep Singh Budania > > > > On Thu, Jul 2, 2020 at 3:53 PM Szalay-Bekő Máté < > szalay.beko.m...@gmail.com> wrote: > >> I think SSL is working for you already... If you managed to start the >> zkCli.sh and be able to connect to ZooKeeper on the secure port and issue >> any kind of command (like: " ls / "), then the wire encryption is working >> and your server/client communication is secured by ZooKeeper. >> >> Why you want to run the following command? >> addauth ztpasswd zooadmin:<password> >> >> Do you also want to configure a superDigest user in ZooKeeper? Please note >> that this command is independent from SSL. If you need to create a >> username-password pair for digest authentication then please use the >> command in the following way: >> addauth digest zooadmin:yourSuperSecretPassword >> >> Kind regards, >> Mate >> >> On Thu, Jul 2, 2020 at 6:59 AM kuldeep singh <kuldeep.sing...@gmail.com> >> wrote: >> >> > 1. sh zkCli.sh --config /etc/zookeeper -server localhost:2281 >> > >> > 2. addauth ztpasswd zooadmin:<password> >> > >> > >> > Thanks, >> > --------------------- >> > Kuldeep Singh Budania >> > >> > >> > >> > On Thu, Jul 2, 2020 at 9:56 AM kuldeep singh <kuldeep.sing...@gmail.com >> > >> > wrote: >> > >> > > Hi Team, >> > > >> > > Any update on this? >> > > >> > > Thanks, >> > > --------------------- >> > > Kuldeep Singh Budania >> > > >> > > >> > > >> > > On Wed, Jul 1, 2020 at 6:43 PM kuldeep singh < >> kuldeep.sing...@gmail.com> >> > > wrote: >> > > >> > >> Sorry this is my bad, there were server setting like below >> > >> >> > >> export SERVER_JVMFLAGS=" >> > >> >> > >> >> > >> >> > >> -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> > >> >> > >> >> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >> > >> -Dzookeeper.ssl.keyStore.password=testpass >> > >> >> > >> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >> > >> -Dzookeeper.ssl.trustStore.password=testpass" >> > >> >> > >> >> > >> >> > >> export CLIENT_JVMFLAGS=" >> > >> >> > >> >> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >> > >> -Dzookeeper.client.secure=true >> > >> >> -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >> > >> -Dzookeeper.ssl.keyStore.password=testpass >> > >> >> > >> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >> > >> -Dzookeeper.ssl.trustStore.password=testpass" >> > >> >> > >> I want to have SSL between client to server communication >> > >> >> > >> I am already following the same link which you have shared with me >> but >> > >> that is not working. >> > >> >> > >> Zoo.cfg >> > >> >> > >> secureClientPort=2281 >> > >> initLimit=10 >> > >> syncLimit=5 >> > >> dataDir=/var/lib/zookeeper/data >> > >> tickTime=2000 >> > >> autopurge.snapRetainCount=3 >> > >> autopurge.purgeInterval=1 >> > >> admin.enableServer=false >> > >> standaloneEnabled=false >> > >> jute.maxbuffer=2147483648 >> > >> serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> > >> server.1=host1_priv:10288:10388 >> > >> server.2=host2_priv:10288:10388 >> > >> server.3=host3_priv:10288:10388 >> > >> >> > >> >> > >> command to connect using zkcli >> > >> >> > >> 1. zkcli zoo.cfg localhost:2281 >> > >> 2. addauth ztpasswd usernaem:password >> > >> >> > >> after second step we are getting below error >> > >> >> > >> WatchedEvent state:AuthFailed type:None path:null >> > >> >> > >> >> > >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN >> > >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication >> > provider >> > >> for scheme: ztpasswd has x509 ip digest* >> > >> >> > >> Thanks, >> > >> --------------------- >> > >> Kuldeep Singh Budania >> > >> >> > >> >> > >> >> > >> On Wed, Jul 1, 2020 at 6:25 PM Szalay-Bekő Máté < >> > >> szalay.beko.m...@gmail.com> wrote: >> > >> >> > >>> > No authentication provider for scheme: ztpasswd has x509 ip >> digest* >> > >>> >> > >>> This suggest you have some configuration error... Where did you use >> the >> > >>> "ztpasswd" string in your configs / commands? >> > >>> >> > >>> On Wed, Jul 1, 2020 at 2:53 PM Szalay-Bekő Máté < >> > >>> szalay.beko.m...@gmail.com> >> > >>> wrote: >> > >>> >> > >>> > > My ZK server is up and running in secure mode >> > >>> > >> > >>> > What is your goal? You want to setup client-server SSL connection? >> > >>> > >> > >>> > see: >> > >>> > >> > >>> >> > >> https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide >> > >>> > >> > >>> > (or you want to have both SSL and SASL enabled?) >> > >>> > >> > >>> > Anyway, please remove the following line from the SERVER_JVMFLAGS: >> > >>> > >> > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >> > >>> > This is a configuration that makes sense only for the ZooKeeper >> > client, >> > >>> > not for the server. For the server, use the following: >> > >>> > >> > >>> > >> > >>> >> > >> -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> > >>> > >> > >>> > If it doesn't solve the issue, then can you please send your >> zoo.cfg >> > >>> file? >> > >>> > Also can you please send the zkCli command you execute? (you need >> to >> > >>> > connect to the secure ZooKeeper port, unless portUnification is >> > >>> enabled) >> > >>> > >> > >>> > Kind regards, >> > >>> > Mate >> > >>> > >> > >>> > On Wed, Jul 1, 2020 at 9:48 AM kuldeep singh < >> > >>> kuldeep.sing...@gmail.com> >> > >>> > wrote: >> > >>> > >> > >>> >> Hi, >> > >>> >> >> > >>> >> we have done below changes in java.env file >> > >>> >> >> > >>> >> export SERVER_JVMFLAGS=" >> > >>> >> >> > >>> >> >> > >>> >> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >> > >>> >> >> > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >> > >>> >> -Dzookeeper.ssl.keyStore.password=testpass >> > >>> >> >> > >>> >> > >> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >> > >>> >> -Dzookeeper.ssl.trustStore.password=testpass" >> > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> export CLIENT_JVMFLAGS=" >> > >>> >> >> > >>> >> >> > >>> >> -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty >> > >>> >> -Dzookeeper.client.secure=true >> > >>> >> >> > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks >> > >>> >> -Dzookeeper.ssl.keyStore.password=testpass >> > >>> >> >> > >>> >> > >> -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks >> > >>> >> -Dzookeeper.ssl.trustStore.password=testpass" >> > >>> >> >> > >>> >> I have started the ZK server and it is up without any issue. >> > >>> >> >> > >>> >> But now when I login to ZkCli then it gives the below error. >> > >>> >> >> > >>> >> WatchedEvent state:AuthFailed type:None path:null >> > >>> >> >> > >>> >> >> > >>> >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN >> > >>> >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No authentication >> > >>> provider >> > >>> >> for scheme: ztpasswd has x509 ip digest* >> > >>> >> >> > >>> >> Please help me on this issue >> > >>> >> >> > >>> >> Thanks, >> > >>> >> --------------------- >> > >>> >> Kuldeep Singh Budania >> > >>> >> Software Architect >> > >>> >> >> > >>> >> >> > >>> >> On Wed, Jul 1, 2020 at 12:05 PM kuldeep singh < >> > >>> kuldeep.sing...@gmail.com> >> > >>> >> wrote: >> > >>> >> >> > >>> >> > Hi, >> > >>> >> > >> > >>> >> > My ZK server is up and running in secure mode, But When I am >> > >>> trying to >> > >>> >> > connect to the ZK server using ZKCli, it gives the below error. >> > >>> >> > >> > >>> >> > WatchedEvent state:AuthFailed type:None path:null >> > >>> >> > >> > >>> >> > >> > >>> >> > Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN >> > >>> >> > [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No >> authentication >> > >>> >> provider >> > >>> >> > for scheme: ztpasswd has x509 ip digest* >> > >>> >> > >> > >>> >> > Can someone please help me on this issue. we are using the >> 3.5.6 >> > >>> >> version. >> > >>> >> > >> > >>> >> > I appreciate if I will get a response as soon as possible, as >> I am >> > >>> stuck >> > >>> >> > at this point. >> > >>> >> > >> > >>> >> > Thanks, >> > >>> >> > --------------------- >> > >>> >> > Kuldeep Singh Budania >> > >>> >> > Software Architect >> > >>> >> > >> > >>> >> > >> > >>> >> > >> > >>> >> > On Thu, Jun 25, 2020 at 11:54 AM Enrico Olivelli - Diennea >> > >>> >> > <enrico.olive...@diennea.com.invalid> wrote: >> > >>> >> > >> > >>> >> >> I mean in zoo.cfg >> > >>> >> >> Not as a system property >> > >>> >> >> >> > >>> >> >> Enrico >> > >>> >> >> >> > >>> >> >> Il giorno 25/06/20, 08:19 "Enrico Olivelli - Diennea" < >> > >>> >> >> enrico.olive...@diennea.com.INVALID> ha scritto: >> > >>> >> >> >> > >>> >> >> Hi >> > >>> >> >> You have to enable Netty on the server side >> > >>> >> >> >> > >>> >> >> Something like: >> > >>> >> >> >> > >>> >> >> > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> > >>> >> >> >> > >>> >> >> Hope that helps >> > >>> >> >> Enrico >> > >>> >> >> >> > >>> >> >> Il giorno 24/06/20, 19:17 "kuldeep singh" < >> > >>> >> kuldeep.sing...@gmail.com> >> > >>> >> >> ha scritto: >> > >>> >> >> >> > >>> >> >> Hi, >> > >>> >> >> >> > >>> >> >> I got below error while setting SSL properties in >> > zkEnv.sh >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> ========================================================================================== >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,864 - INFO >> > [main:QuorumPeerConfig@133] >> > >>> - >> > >>> >> >> Reading >> > >>> >> >> configuration from: /etc/zookeeper/zoo.cfg >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,874 - INFO >> > [main:QuorumPeerConfig@385] >> > >>> - >> > >>> >> >> clientPortAddress is 0.0.0.0/0.0.0.0:10181 >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,874 - INFO >> > [main:QuorumPeerConfig@399] >> > >>> - >> > >>> >> >> secureClientPortAddress is 0.0.0.0/0.0.0.0:2281 >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,878 - INFO [main:X509Util@79] - >> > >>> Setting >> > >>> >> -D >> > >>> >> >> jdk.tls.rejectClientInitiatedRenegotiation=true to >> > disable >> > >>> >> >> client-initiated >> > >>> >> >> TLS renegotiation >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,897 - INFO >> > >>> [main:DatadirCleanupManager@78] >> > >>> >> - >> > >>> >> >> autopurge.snapRetainCount set to 3 >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,897 - INFO >> > >>> [main:DatadirCleanupManager@79] >> > >>> >> - >> > >>> >> >> autopurge.purgeInterval set to 1 >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,898 - INFO [ >> > >>> >> >> PurgeTask:DatadirCleanupManager$PurgeTask@138] - >> Purge >> > >>> task >> > >>> >> >> started. >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,899 - INFO [main:ManagedUtil@46] >> - >> > >>> Log4j >> > >>> >> >> found with >> > >>> >> >> jmx enabled. >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,903 - INFO >> > >>> [PurgeTask:FileTxnSnapLog@103] >> > >>> >> - >> > >>> >> >> zookeeper.snapshot.trust.empty : false >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,910 - INFO [ >> > >>> >> >> PurgeTask:DatadirCleanupManager$PurgeTask@144] - >> Purge >> > >>> task >> > >>> >> >> completed. >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,975 - INFO >> [main:QuorumPeerMain@141 >> > ] >> > >>> - >> > >>> >> >> Starting quorum >> > >>> >> >> peer >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,983 - INFO >> > [main:ServerCnxnFactory@135] >> > >>> - >> > >>> >> >> Using >> > >>> >> >> org.apache.zookeeper.server.NIOServerCnxnFactory as >> > server >> > >>> >> >> connection >> > >>> >> >> factory >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,986 - INFO >> > >>> [main:NIOServerCnxnFactory@673] >> > >>> >> - >> > >>> >> >> Configuring NIO connection handler with 10s >> sessionless >> > >>> >> >> connection timeout, >> > >>> >> >> 2 selector thread(s), 16 worker threads, and 64 kB >> direct >> > >>> >> buffers. >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,992 - INFO >> > >>> [main:NIOServerCnxnFactory@686 >> > >>> >> ] >> > >>> >> >> - binding >> > >>> >> >> to port 0.0.0.0/0.0.0.0:10181 >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,994 - INFO >> > [main:ServerCnxnFactory@135] >> > >>> - >> > >>> >> >> Using >> > >>> >> >> org.apache.zookeeper.server.NIOServerCnxnFactory as >> > server >> > >>> >> >> connection >> > >>> >> >> factory >> > >>> >> >> >> > >>> >> >> 2020-06-24 15:49:35,995 - ERROR >> [main:QuorumPeerMain@101 >> > ] >> > >>> - >> > >>> >> >> Unexpected >> > >>> >> >> exception, exiting abnormally >> > >>> >> >> >> > >>> >> >> java.lang.UnsupportedOperationException: SSL isn't >> > >>> supported in >> > >>> >> >> NIOServerCnxn >> > >>> >> >> >> > >>> >> >> at >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:644) >> > >>> >> >> >> > >>> >> >> at >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:155) >> > >>> >> >> >> > >>> >> >> at >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) >> > >>> >> >> >> > >>> >> >> at >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> I have set the following properties in >> SERVER_JVMFLAGS in >> > >>> >> >> zkEnv.sh file : >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> "-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> -Dzookeeper.ssl.keyStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkkeystore.p12 >> > >>> >> >> -Dzookeeper.ssl.keyStore.password=EvaiKiO1@123456 >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> > >>> >> > >> -Dzookeeper.ssl.trustStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkTrustStore.jks >> > >>> >> >> -Dzookeeper.ssl.trustStore.password=EvaiKiO1@123456" >> > >>> >> >> >> > >>> >> >> Thanks, >> > >>> >> >> --------------------- >> > >>> >> >> Kuldeep Singh Budania >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> On Mon, Jun 22, 2020 at 8:08 PM Jordan Zimmerman < >> > >>> >> >> jor...@jordanzimmerman.com> >> > >>> >> >> wrote: >> > >>> >> >> >> > >>> >> >> > It's the same as the normal ZooKeeper client: >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions >> > >>> >> >> > < >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions >> > >>> >> >> > > >> > >>> >> >> > >> > >>> >> >> > -Jordan >> > >>> >> >> > >> > >>> >> >> > > On Jun 22, 2020, at 5:50 AM, kuldeep singh < >> > >>> >> >> kuldeep.sing...@gmail.com> >> > >>> >> >> > wrote: >> > >>> >> >> > > >> > >>> >> >> > > Hi Team, >> > >>> >> >> > > How we will do secure communication between the >> > Curator >> > >>> >> >> framework and >> > >>> >> >> > > zookeeper 3.5.6 ? >> > >>> >> >> > > I didn't get any solution right now. >> > >>> >> >> > > I appreciate it if someone could help me with the >> > same. >> > >>> >> >> > > >> > >>> >> >> > > Thanks, >> > >>> >> >> > > --------------------- >> > >>> >> >> > > Kuldeep Singh Budania >> > >>> >> >> > > Software Architect >> > >>> >> >> > > >> > >>> >> >> > > >> > >>> >> >> > > On Fri, Apr 17, 2020 at 4:53 PM Szalay-Bekő Máté < >> > >>> >> >> > szalay.beko.m...@gmail.com> >> > >>> >> >> > > wrote: >> > >>> >> >> > > >> > >>> >> >> > >> Hello Kuldeep, >> > >>> >> >> > >> >> > >>> >> >> > >> did you download the source from the ZooKeeper >> > >>> webpage, or >> > >>> >> >> checked out >> > >>> >> >> > from >> > >>> >> >> > >> git? >> > >>> >> >> > >> Anyway, the following commands should work: >> > >>> >> >> > >> >> > >>> >> >> > >> wget >> > >>> >> >> > >> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> https://downloads.apache.org/zookeeper/zookeeper-3.5.7/apache-zookeeper-3.5.7.tar.gz >> > >>> >> >> > >> tar xzvf ./apache-zookeeper-3.5.7.tar.gz >> > >>> >> >> > >> cd apache-zookeeper-3.5.7 >> > >>> >> >> > >> mvn clean install -DskipTests >> > >>> >> >> > >> >> > >>> >> >> > >> I tested it with OpenJDK 8u424 and maven 3.6.0. >> > >>> >> >> > >> >> > >>> >> >> > >> Kind regards, >> > >>> >> >> > >> Mate >> > >>> >> >> > >> >> > >>> >> >> > >> >> > >>> >> >> > >> On Fri, Apr 17, 2020 at 12:51 PM kuldeep singh < >> > >>> >> >> > kuldeep.sing...@gmail.com> >> > >>> >> >> > >> wrote: >> > >>> >> >> > >> >> > >>> >> >> > >>> Hi All, >> > >>> >> >> > >>> >> > >>> >> >> > >>> I have download the zookeeper source code and >> run >> > >>> the mvn >> > >>> >> >> command for >> > >>> >> >> > >>> packaging but get problem on "zookeeper-server" >> > >>> >> directory. >> > >>> >> >> > >>> >> > >>> >> >> > >>> *command :- mvn clean package* >> > >>> >> >> > >>> >> > >>> >> >> > >>> [ERROR] Failed to execute goal >> > >>> >> >> > >>> >> > >>> >> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile >> > >>> >> >> > >>> (default-compile) on project zookeeper: Fatal >> error >> > >>> >> >> compiling: >> > >>> >> >> > >>> java.lang.NullPointerException -> [Help 1] >> > >>> >> >> > >>> >> > >>> org.apache.maven.lifecycle.LifecycleExecutionException: >> > >>> >> >> Failed to >> > >>> >> >> > execute >> > >>> >> >> > >>> goal >> > >>> >> >> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile >> > >>> >> >> > >>> (default-compile) on project zookeeper: Fatal >> error >> > >>> >> >> compiling >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) >> > >>> >> >> > >>> at >> > >>> >> >> > >> > >>> >> org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) >> > >>> >> >> > >>> at >> > >>> >> >> org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) >> > >>> >> >> > >>> at >> > >>> >> >> org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) >> > >>> >> >> > >>> at >> > >>> >> >> org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) >> > >>> >> >> > >>> at >> > >>> >> >> org.apache.maven.cli.MavenCli.main(MavenCli.java:157) >> > >>> >> >> > >>> at >> > >>> >> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > >>> >> >> > >>> at >> > >>> >> java.lang.reflect.Method.invoke(Method.java:498) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) >> > >>> >> >> > >>> Caused by: >> > >>> >> org.apache.maven.plugin.MojoExecutionException: >> > >>> >> >> Fatal error >> > >>> >> >> > >>> compiling >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1140) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:188) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) >> > >>> >> >> > >>> ... 19 more >> > >>> >> >> > >>> Caused by: >> > >>> >> org.codehaus.plexus.compiler.CompilerException: >> > >>> >> >> > >>> java.lang.NullPointerException >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:173) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1129) >> > >>> >> >> > >>> ... 22 more >> > >>> >> >> > >>> Caused by: java.lang.RuntimeException: >> > >>> >> >> java.lang.NullPointerException >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.tools.javac.main.Main.compile(Main.java:559) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126) >> > >>> >> >> > >>> ... 24 more >> > >>> >> >> > >>> Caused by: java.lang.NullPointerException >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.javac.model.JavacTypes.getOverriddenMethods(JavacTypes.java:306) >> > >>> >> >> > >>> at >> > >>> >> com.sun.tools.doclint.Env.setCurrent(Env.java:155) >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.tools.doclint.Checker.scan(Checker.java:144) >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.tools.doclint.DocLint$2.visitDecl(DocLint.java:285) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:359) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.visitNewClass(TreeScanner.java:280) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.tools.javac.tree.JCTree$JCNewClass.accept(JCTree.java:1532) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.visitVariable(TreeScanner.java:153) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:373) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:346) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:864) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) >> > >>> >> >> > >>> at >> > >>> >> >> > >> >> > >>> >> >> >> com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:133) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:360) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:120) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:354) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:346) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:550) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.tools.doclint.DocLint$3.started(DocLint.java:296) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.started(ClientCodeWrapper.java:668) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> com.sun.tools.javac.api.MultiTaskListener.started(MultiTaskListener.java:103) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1240) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> >> com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901) >> > >>> >> >> > >>> at >> > >>> >> >> > >>> >> > >>> >> >> >> > >>> com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860) >> > >>> >> >> > >>> at >> > >>> >> >> com.sun.tools.javac.main.Main.compile(Main.java:523) >> > >>> >> >> > >>> ... 27 more >> > >>> >> >> > >>> [ERROR] >> > >>> >> >> > >>> [ERROR] Re-run Maven using the -X switch to >> enable >> > >>> full >> > >>> >> >> debug logging. >> > >>> >> >> > >>> [ERROR] >> > >>> >> >> > >>> [ERROR] For more information about the errors >> and >> > >>> >> possible >> > >>> >> >> solutions, >> > >>> >> >> > >>> please read the following articles: >> > >>> >> >> > >>> [ERROR] [Help 1] >> > >>> >> >> > >>> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException >> > >>> >> >> > >>> >> > >>> >> >> > >>> Can someone help me on this issue? >> > >>> >> >> > >>> >> > >>> >> >> > >>> Thanks, >> > >>> >> >> > >>> --------------------- >> > >>> >> >> > >>> Kuldeep Singh Budania >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >>> >> > >>> >> >> > >>> On Sat, Apr 4, 2020 at 5:57 PM Szalay-Bekő Máté >> < >> > >>> >> >> > >>> szalay.beko.m...@gmail.com> >> > >>> >> >> > >>> wrote: >> > >>> >> >> > >>> >> > >>> >> >> > >>>> these exceptions can mean many things... I >> think >> > >>> this >> > >>> >> can >> > >>> >> >> be even >> > >>> >> >> > >> normal >> > >>> >> >> > >>>> duding rolling restart (as some connections get >> > >>> broken >> > >>> >> in >> > >>> >> >> this case) >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> However, I saw cases already when exceptions >> like >> > >>> these >> > >>> >> >> killed >> > >>> >> >> > receiver >> > >>> >> >> > >>> or >> > >>> >> >> > >>>> sender threads in QuorumCnxManager / Leader >> > >>> Election in >> > >>> >> >> such a way >> > >>> >> >> > that >> > >>> >> >> > >>>> they were not able to recover, so the node was >> > >>> unable to >> > >>> >> >> connect to >> > >>> >> >> > any >> > >>> >> >> > >>>> quorum until restart. I remember seeing this in >> > 3.4 >> > >>> too. >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> Do you see these exceptions in the second >> server >> > >>> (the >> > >>> >> one >> > >>> >> >> which you >> > >>> >> >> > >> just >> > >>> >> >> > >>>> upgraded in step 3)? >> > >>> >> >> > >>>> Is this issue reproducible? >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> What is the tickTime and initLimit you use? >> Maybe >> > >>> the >> > >>> >> >> server just >> > >>> >> >> > >> require >> > >>> >> >> > >>>> more time to sync? >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> I would need more logs to really see what >> > happened. >> > >>> Can >> > >>> >> >> you create a >> > >>> >> >> > >> Jira >> > >>> >> >> > >>>> issue and upload the logs and also the >> ZooKeeper >> > >>> >> configs? >> > >>> >> >> I am happy >> > >>> >> >> > to >> > >>> >> >> > >>>> take a closer look. >> > >>> >> >> > >>>> (if you need to re-run the test to collect the >> > logs, >> > >>> >> then >> > >>> >> >> enabling >> > >>> >> >> > >> DEBUG >> > >>> >> >> > >>>> logs would be great. The INFO level logs are >> > usually >> > >>> >> >> enough for these >> > >>> >> >> > >>>> problems, but one can never know...) >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> Kind regards, >> > >>> >> >> > >>>> Mate >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> On Fri, Apr 3, 2020 at 10:05 AM kuldeep singh < >> > >>> >> >> > >> kuldeep.sing...@gmail.com >> > >>> >> >> > >>>> >> > >>> >> >> > >>>> wrote: >> > >>> >> >> > >>>> >> > >>> >> >> > >>>>> Hi Team, >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> I have done some POC on rolling upgrade and >> found >> > >>> below >> > >>> >> >> result. >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> 1. On 1st node upgrade zookeeper . Traffic >> was >> > >>> >> running >> > >>> >> >> fine >> > >>> >> >> > >> because >> > >>> >> >> > >>> 2 >> > >>> >> >> > >>>>> nodes are already on old zookeeper. >> > >>> >> >> > >>>>> 2. On 1st node upgrade our application and >> > didn’t >> > >>> >> find >> > >>> >> >> any issue >> > >>> >> >> > >>>>> 3. On 2nd node upgrade zookeeper but got >> below >> > >>> error >> > >>> >> >> and zookeeper >> > >>> >> >> > >>> is >> > >>> >> >> > >>>>> not taking any requests >> > >>> >> >> > >>>>> 4. >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> java.io.EOFException >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> java.io.DataInputStream.readInt(DataInputStream.java:392) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:747) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> 2020-03-30 14:19:55,587 - WARN >> > >>> >> >> > >>>>> [RecvWorker:1:QuorumCnxManager$RecvWorker@765] >> - >> > >>> >> >> Interrupting >> > >>> >> >> > >>> SendWorker >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> 2020-03-30 14:19:55,588 - ERROR >> [LearnerHandler-/ >> > >>> >> >> 192.168.44.73:33754 >> > >>> >> >> > >>>>> :LearnerHandler@562] - Unexpected exception >> > >>> causing >> > >>> >> >> shutdown while >> > >>> >> >> > >>> sock >> > >>> >> >> > >>>>> still open >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> java.io.EOFException >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> java.io.DataInputStream.readInt(DataInputStream.java:392) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> > >>>>> >> > >>> >> >> > >> >> > >>> >> >> >> > >>> >> org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> at >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> > >>> >> > >> org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:476) >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> 2020-03-30 14:19:55,588 - WARN >> > >>> >> >> > >>>>> [SendWorker:1:QuorumCnxManager$SendWorker@679] >> - >> > >>> >> >> Interrupted while >> > >>> >> >> > >>>> waiting >> > >>> >> >> > >>>>> for message on queue >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> Please let me know is this the known issue or >> > this >> > >>> is >> > >>> >> >> different issue >> > >>> >> >> > >>>> which >> > >>> >> >> > >>>>> is mention in Apache zookeeper documentation >> when >> > >>> >> >> upgrading from >> > >>> >> >> > >> 3.4.5 >> > >>> >> >> > >>> to >> > >>> >> >> > >>>>> 3.5.6 >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> Thanks, >> > >>> >> >> > >>>>> --------------------- >> > >>> >> >> > >>>>> Kuldeep Singh Budania >> > >>> >> >> > >>>>> Software Architect >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>> On Sun, Mar 29, 2020 at 9:06 AM Alexander >> Shraer >> > < >> > >>> >> >> shra...@gmail.com> >> > >>> >> >> > >>>>> wrote: >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>>>> +1 to what Mate said (I wrote the quoted >> > >>> >> instructions). >> > >>> >> >> > >>>>>> >> > >>> >> >> > >>>>>> >> > >>> >> >> > >>>>>> >> > >>> >> >> > >>>>>> On Tue, Mar 24, 2020 at 7:03 AM Szalay-Bekő >> > Máté < >> > >>> >> >> > >>>>>> szalay.beko.m...@gmail.com> >> > >>> >> >> > >>>>>> wrote: >> > >>> >> >> > >>>>>> >> > >>> >> >> > >>>>>>> Hi Kuldeep, >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> I just want to provide you some background >> info >> > >>> about >> > >>> >> >> our >> > >>> >> >> > >>>>> documentation. >> > >>> >> >> > >>>>>>> The reason to upgrade to 3.4.6 first is to >> > avoid >> > >>> the >> > >>> >> >> following >> > >>> >> >> > >>> error: >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN >> > >>> [localhost/ >> > >>> >> >> > >>> 127.0.0.1:2784 >> > >>> >> >> > >>>>>>> :QuorumCnxManager@349] - Invalid server id: >> > >>> -65536 >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> This error comes because of the protocol >> > changes >> > >>> >> between >> > >>> >> >> > >> ZooKeeper >> > >>> >> >> > >>>>> server >> > >>> >> >> > >>>>>>> nodes during connection initiation for >> leader >> > >>> >> election. >> > >>> >> >> In >> > >>> >> >> > >>> ZooKeeper >> > >>> >> >> > >>>>> 3.5 >> > >>> >> >> > >>>>>> a >> > >>> >> >> > >>>>>>> protocol version was introduced (see >> > >>> ZOOKEEPER-107) >> > >>> >> and >> > >>> >> >> since >> > >>> >> >> > >> that >> > >>> >> >> > >>>> time >> > >>> >> >> > >>>>>> the >> > >>> >> >> > >>>>>>> fist long value sent in the initial message >> is >> > >>> not >> > >>> >> the >> > >>> >> >> server ID >> > >>> >> >> > >>> but >> > >>> >> >> > >>>>> the >> > >>> >> >> > >>>>>>> protocol version (-65536). In ZooKeeper >> 3.4.6 >> > we >> > >>> made >> > >>> >> >> the old 3.4 >> > >>> >> >> > >>>>>>> ZooKeepers backward compatible, so they are >> > able >> > >>> to >> > >>> >> >> parse both >> > >>> >> >> > >> the >> > >>> >> >> > >>>> old >> > >>> >> >> > >>>>>> and >> > >>> >> >> > >>>>>>> the new protocol format (see >> ZOOKEEPER-1633). >> > >>> This >> > >>> >> >> issue happens >> > >>> >> >> > >>> only >> > >>> >> >> > >>>>>> when >> > >>> >> >> > >>>>>>> you need to use old (3.4.0 - 3.4.5) and new >> > >>> (3.5.0+) >> > >>> >> >> ZooKeeper >> > >>> >> >> > >>>> servers >> > >>> >> >> > >>>>>>> together in the same cluster. During a >> rolling >> > >>> >> upgrade, >> > >>> >> >> this is >> > >>> >> >> > >>>> usually >> > >>> >> >> > >>>>>> the >> > >>> >> >> > >>>>>>> case to have old and new ZooKeepers present >> > >>> together. >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> The fact that you haven't seen any issues >> might >> > >>> be >> > >>> >> >> caused by the >> > >>> >> >> > >>>> order >> > >>> >> >> > >>>>> of >> > >>> >> >> > >>>>>>> the servers. In ZooKeeper the connection >> > >>> initiation >> > >>> >> >> between the >> > >>> >> >> > >>>> servers >> > >>> >> >> > >>>>>>> during the leader election follows a >> specific >> > >>> rule. >> > >>> >> As >> > >>> >> >> far as I >> > >>> >> >> > >>>>> remember >> > >>> >> >> > >>>>>>> always the server with the larger ID 'wins >> the >> > >>> >> >> challenge', so it >> > >>> >> >> > >> is >> > >>> >> >> > >>>>>>> possible, that the old server didn't need to >> > >>> parse >> > >>> >> any >> > >>> >> >> initial >> > >>> >> >> > >>>> message >> > >>> >> >> > >>>>>> (if >> > >>> >> >> > >>>>>>> it had the largest ID) and this is why you >> > >>> haven't >> > >>> >> seen >> > >>> >> >> the >> > >>> >> >> > >> issue. >> > >>> >> >> > >>>> Also >> > >>> >> >> > >>>>>>> having 2 nodes up from the 3 nodes cluster >> > still >> > >>> >> makes >> > >>> >> >> the >> > >>> >> >> > >> cluster >> > >>> >> >> > >>>> work >> > >>> >> >> > >>>>>> (so >> > >>> >> >> > >>>>>>> you should also check if all the servers are >> > >>> part of >> > >>> >> >> the quorum). >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> I agree with Enrico and Norbert, the safest >> and >> > >>> most >> > >>> >> >> stable way >> > >>> >> >> > >> is >> > >>> >> >> > >>>>>> upgrade >> > >>> >> >> > >>>>>>> first to 3.4.latest, then go to 3.5.latest. >> > >>> Still, if >> > >>> >> >> you don't >> > >>> >> >> > >> see >> > >>> >> >> > >>>>> that >> > >>> >> >> > >>>>>>> you would hit this specific issue (e.g. no >> > >>> "Invalid >> > >>> >> >> server id" in >> > >>> >> >> > >>> the >> > >>> >> >> > >>>>> log >> > >>> >> >> > >>>>>>> files), and all the three servers can handle >> > >>> traffic, >> > >>> >> >> then maybe >> > >>> >> >> > >>> you >> > >>> >> >> > >>>>>> don't >> > >>> >> >> > >>>>>>> need to upgrade first to 3.4.latest, it is >> your >> > >>> >> >> decision. >> > >>> >> >> > >>> Definitely >> > >>> >> >> > >>>>> you >> > >>> >> >> > >>>>>>> should test it first, as suggested by the >> > others. >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> Kind regards, >> > >>> >> >> > >>>>>>> Mate >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> On Tue, Mar 24, 2020 at 12:29 PM Norbert >> Kalmar >> > >>> >> >> > >>>>>>> <nkal...@cloudera.com.invalid> wrote: >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>>> Hi, >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>>> That guide is to upgrade to 3.5.0, which >> was >> > an >> > >>> >> alpha >> > >>> >> >> version. >> > >>> >> >> > >> A >> > >>> >> >> > >>>> lot >> > >>> >> >> > >>>>>> has >> > >>> >> >> > >>>>>>>> changed for the first stable release of >> 3.5.5 >> > >>> and >> > >>> >> then >> > >>> >> >> a few >> > >>> >> >> > >>> more, >> > >>> >> >> > >>>>> even >> > >>> >> >> > >>>>>>>> rolling upgrade issues have been fixed for >> > >>> 3.5.6. >> > >>> >> >> > >>>>>>>> This is a more up-to-date guide: >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>> >> > >>> >> >> >> > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>>> If you have done your testing (with prod >> > >>> snapshot!), >> > >>> >> >> then you >> > >>> >> >> > >> can >> > >>> >> >> > >>>>> skip >> > >>> >> >> > >>>>>>> 3.4 >> > >>> >> >> > >>>>>>>> latest upgrade, but keep in mind we do our >> > >>> >> >> recommendations for >> > >>> >> >> > >> a >> > >>> >> >> > >>>>>> reason. >> > >>> >> >> > >>>>>>>> There were issues reported and/or found >> during >> > >>> >> >> testing. Some >> > >>> >> >> > >> are >> > >>> >> >> > >>>>> fixed >> > >>> >> >> > >>>>>>> with >> > >>> >> >> > >>>>>>>> 3.5.6, some only happens if certain >> conditions >> > >>> stand >> > >>> >> >> > >>> (IOException: >> > >>> >> >> > >>>> No >> > >>> >> >> > >>>>>>>> snapshot found - mentioned in the guide, >> fixed >> > >>> in >> > >>> >> >> 3.5.6). >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>>> So it is up to you, I would still >> recommend to >> > >>> do an >> > >>> >> >> 3.4 >> > >>> >> >> > >> upgrade >> > >>> >> >> > >>>>> first, >> > >>> >> >> > >>>>>>> if >> > >>> >> >> > >>>>>>>> it's feasible. >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>>> Regards, >> > >>> >> >> > >>>>>>>> Norbert >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>>> On Tue, Mar 24, 2020 at 11:45 AM kuldeep >> > singh < >> > >>> >> >> > >>>>>>> kuldeep.sing...@gmail.com> >> > >>> >> >> > >>>>>>>> wrote: >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>>>> Hi, >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> Current Zookeeper version :- 3.4.5 >> > >>> >> >> > >>>>>>>>> Upgraded version :- 3.5.6 >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> We are not going with 3.5.7. Our final >> > >>> decision is >> > >>> >> >> zookeeper >> > >>> >> >> > >>>>> version >> > >>> >> >> > >>>>>> is >> > >>> >> >> > >>>>>>>>> 3.5.6 >> > >>> >> >> > >>>>>>>>> as per your reply first we need to move >> > latest >> > >>> >> >> version of >> > >>> >> >> > >>> 3.4.x, >> > >>> >> >> > >>>>> like >> > >>> >> >> > >>>>>>>> below >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> 3.4.5 -> 3.4.14 -> 3.5.6 (Correct me if I >> am >> > >>> wrong >> > >>> >> >> here) >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> But if We are not facing any problem that >> i >> > >>> have >> > >>> >> >> shared you >> > >>> >> >> > >>> that >> > >>> >> >> > >>>> we >> > >>> >> >> > >>>>>>> have >> > >>> >> >> > >>>>>>>>> set up of 3 node cluster where 2 node are >> on >> > >>> 3.5.6 >> > >>> >> >> version >> > >>> >> >> > >> and >> > >>> >> >> > >>> 1 >> > >>> >> >> > >>>>> node >> > >>> >> >> > >>>>>>> on >> > >>> >> >> > >>>>>>>>> 3.4.5, Everything is running fine and >> didn't >> > >>> get >> > >>> >> any >> > >>> >> >> issue, >> > >>> >> >> > >> So >> > >>> >> >> > >>>> what >> > >>> >> >> > >>>>>>> other >> > >>> >> >> > >>>>>>>>> problem we can face if we directly move to >> > >>> 3.5.6 >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> Thanks, >> > >>> >> >> > >>>>>>>>> --------------------- >> > >>> >> >> > >>>>>>>>> Kuldeep Singh Budania >> > >>> >> >> > >>>>>>>>> Software Architect >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>> On Tue, Mar 24, 2020 at 3:58 PM Enrico >> > >>> Olivelli < >> > >>> >> >> > >>>>> eolive...@gmail.com >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>>>> wrote: >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>>>> Hi >> > >>> >> >> > >>>>>>>>>> You have to upgrade to latest 3.4.x >> > Zookeeper >> > >>> then >> > >>> >> >> you will >> > >>> >> >> > >>>>> upgrade >> > >>> >> >> > >>>>>>> to >> > >>> >> >> > >>>>>>>>>> 3.5.7. >> > >>> >> >> > >>>>>>>>>> All should run well without issues >> > >>> >> >> > >>>>>>>>>> >> > >>> >> >> > >>>>>>>>>> >> > >>> >> >> > >>>>>>>>>> Enrico >> > >>> >> >> > >>>>>>>>>> >> > >>> >> >> > >>>>>>>>>> Il Mar 24 Mar 2020, 10:18 kuldeep singh < >> > >>> >> >> > >>>>> kuldeep.sing...@gmail.com >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>>> ha >> > >>> >> >> > >>>>>>>>>> scritto: >> > >>> >> >> > >>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> Hi Team, >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> We are upgrading zookeeper from 3.4.5 to >> > >>> 3.5.6. I >> > >>> >> >> have >> > >>> >> >> > >> set >> > >>> >> >> > >>>> up 3 >> > >>> >> >> > >>>>>>> node >> > >>> >> >> > >>>>>>>>>>> cluster where 2 node are on 3.5.6 >> version >> > >>> and 1 >> > >>> >> >> node on >> > >>> >> >> > >>>> 3.4.5. >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> Everything is running fine and didn't >> get >> > any >> > >>> >> issue >> > >>> >> >> on my >> > >>> >> >> > >>>>> system. >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> but I found something on apache site >> that >> > >>> first >> > >>> >> we >> > >>> >> >> need >> > >>> >> >> > >> to >> > >>> >> >> > >>>>>> upgrade >> > >>> >> >> > >>>>>>>> on >> > >>> >> >> > >>>>>>>>>>> 3.4.6 than we can upgrade to 3.5.6. So >> is >> > it >> > >>> >> >> mandatory >> > >>> >> >> > >> to >> > >>> >> >> > >>> go >> > >>> >> >> > >>>>> on >> > >>> >> >> > >>>>>>>> 3.4.6 >> > >>> >> >> > >>>>>>>>>>> first. >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> *Upgrading to 3.5.0* >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> Upgrading a running ZooKeeper ensemble >> to >> > >>> 3.5.0 >> > >>> >> >> should be >> > >>> >> >> > >>>> done >> > >>> >> >> > >>>>>> only >> > >>> >> >> > >>>>>>>>> after >> > >>> >> >> > >>>>>>>>>>> upgrading your ensemble to the 3.4.6 >> > release. >> > >>> >> Note >> > >>> >> >> that >> > >>> >> >> > >>> this >> > >>> >> >> > >>>> is >> > >>> >> >> > >>>>>>> only >> > >>> >> >> > >>>>>>>>>>> necessary for rolling upgrades (if >> you're >> > >>> fine >> > >>> >> with >> > >>> >> >> > >>> shutting >> > >>> >> >> > >>>>> down >> > >>> >> >> > >>>>>>> the >> > >>> >> >> > >>>>>>>>>>> system completely, you don't have to go >> > >>> through >> > >>> >> >> 3.4.6). >> > >>> >> >> > >> If >> > >>> >> >> > >>>> you >> > >>> >> >> > >>>>>>>> attempt >> > >>> >> >> > >>>>>>>>> a >> > >>> >> >> > >>>>>>>>>>> rolling upgrade without going through >> 3.4.6 >> > >>> (for >> > >>> >> >> example >> > >>> >> >> > >>> from >> > >>> >> >> > >>>>>>> 3.4.5), >> > >>> >> >> > >>>>>>>>> you >> > >>> >> >> > >>>>>>>>>>> may get the following error: >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - INFO >> > >>> >> [localhost/ >> > >>> >> >> > >>>>>> 127.0.0.1:2784 >> > >>> >> >> > >>>>>>>>>>> :QuorumCnxManager$Listener@498] - >> Received >> > >>> >> >> connection >> > >>> >> >> > >>>> request >> > >>> >> >> > >>>>> / >> > >>> >> >> > >>>>>>>>>>> 127.0.0.1:60876 >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 [myid:2] - WARN >> > >>> >> [localhost/ >> > >>> >> >> > >>>>>> 127.0.0.1:2784 >> > >>> >> >> > >>>>>>>>>>> :QuorumCnxManager@349] - Invalid server >> > id: >> > >>> >> -65536 >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> During a rolling upgrade, each server is >> > >>> taken >> > >>> >> down >> > >>> >> >> in >> > >>> >> >> > >> turn >> > >>> >> >> > >>>> and >> > >>> >> >> > >>>>>>>>> rebooted >> > >>> >> >> > >>>>>>>>>>> with the new 3.5.0 binaries. Before >> > starting >> > >>> the >> > >>> >> >> server >> > >>> >> >> > >>> with >> > >>> >> >> > >>>>>> 3.5.0 >> > >>> >> >> > >>>>>>>>>>> binaries, we highly recommend updating >> the >> > >>> >> >> configuration >> > >>> >> >> > >>> file >> > >>> >> >> > >>>>> so >> > >>> >> >> > >>>>>>> that >> > >>> >> >> > >>>>>>>>> all >> > >>> >> >> > >>>>>>>>>>> server statements "server.x=..." contain >> > >>> client >> > >>> >> >> ports >> > >>> >> >> > >> (see >> > >>> >> >> > >>>> the >> > >>> >> >> > >>>>>>>> section >> > >>> >> >> > >>>>>>>>>>> Specifying >> > >>> >> >> > >>>>>>>>>>> the client port). As explained earlier >> you >> > >>> may >> > >>> >> >> leave the >> > >>> >> >> > >>>>>>>> configuration >> > >>> >> >> > >>>>>>>>>> in a >> > >>> >> >> > >>>>>>>>>>> single file, as well as leave the >> > >>> >> >> > >>>> clientPort/clientPortAddress >> > >>> >> >> > >>>>>>>>> statements >> > >>> >> >> > >>>>>>>>>>> (although if you specify client ports in >> > the >> > >>> new >> > >>> >> >> format, >> > >>> >> >> > >>>> these >> > >>> >> >> > >>>>>>>>> statements >> > >>> >> >> > >>>>>>>>>>> are now redundant). >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> Could you please let me know about this >> > case. >> > >>> >> >> Appreciate >> > >>> >> >> > >> if >> > >>> >> >> > >>>>>> respond >> > >>> >> >> > >>>>>>>>> soon. >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>>> Thanks, >> > >>> >> >> > >>>>>>>>>>> --------------------- >> > >>> >> >> > >>>>>>>>>>> Kuldeep Singh Budania >> > >>> >> >> > >>>>>>>>>>> >> > >>> >> >> > >>>>>>>>>> >> > >>> >> >> > >>>>>>>>> >> > >>> >> >> > >>>>>>>> >> > >>> >> >> > >>>>>>> >> > >>> >> >> > >>>>>> >> > >>> >> >> > >>>>> >> > >>> >> >> > >>>> >> > >>> >> >> > >>> >> > >>> >> >> > >> >> > >>> >> >> > >> > >>> >> >> > >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> ________________________________ >> > >>> >> >> >> > >>> >> >> CONFIDENTIALITY & PRIVACY NOTICE >> > >>> >> >> This e-mail (including any attachments) is strictly >> > >>> confidential >> > >>> >> and >> > >>> >> >> may also contain privileged information. If you are not the >> > >>> intended >> > >>> >> >> recipient you are not authorised to read, print, save, >> process or >> > >>> >> disclose >> > >>> >> >> this message. If you have received this message by mistake, >> > please >> > >>> >> inform >> > >>> >> >> the sender immediately and destroy this e-mail, its >> attachments >> > >>> and any >> > >>> >> >> copies. Any use, distribution, reproduction or disclosure by >> any >> > >>> person >> > >>> >> >> other than the intended recipient is strictly prohibited and >> the >> > >>> person >> > >>> >> >> responsible may incur in penalties. >> > >>> >> >> The use of this e-mail is only for professional purposes; >> > >>> there is >> > >>> >> no >> > >>> >> >> guarantee that the correspondence towards this e-mail will be >> > read >> > >>> >> only by >> > >>> >> >> the recipient, because, under certain circumstances, there may >> > be a >> > >>> >> need to >> > >>> >> >> access this email by third subjects belonging to the Company. >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> ________________________________ >> > >>> >> >> >> > >>> >> >> CONFIDENTIALITY & PRIVACY NOTICE >> > >>> >> >> This e-mail (including any attachments) is strictly >> confidential >> > >>> and >> > >>> >> may >> > >>> >> >> also contain privileged information. If you are not the >> intended >> > >>> >> recipient >> > >>> >> >> you are not authorised to read, print, save, process or >> disclose >> > >>> this >> > >>> >> >> message. If you have received this message by mistake, please >> > >>> inform >> > >>> >> the >> > >>> >> >> sender immediately and destroy this e-mail, its attachments >> and >> > any >> > >>> >> copies. >> > >>> >> >> Any use, distribution, reproduction or disclosure by any >> person >> > >>> other >> > >>> >> than >> > >>> >> >> the intended recipient is strictly prohibited and the person >> > >>> >> responsible >> > >>> >> >> may incur in penalties. >> > >>> >> >> The use of this e-mail is only for professional purposes; >> there >> > is >> > >>> no >> > >>> >> >> guarantee that the correspondence towards this e-mail will be >> > read >> > >>> >> only by >> > >>> >> >> the recipient, because, under certain circumstances, there may >> > be a >> > >>> >> need to >> > >>> >> >> access this email by third subjects belonging to the Company. >> > >>> >> >> >> > >>> >> > >> > >>> >> >> > >>> > >> > >>> >> > >> >> > >> >
