Sounds like a bug or a configuration issue... can you share the configs (before and after the scale-up) and the logs? also: does the truststores recognise all the keys used on all the 5 nodes? (e.g. the truststores on the old nodes accept the new keys?)
Best Regards, Mate On Thu, Oct 8, 2020 at 2:31 PM kuldeep singh <kuldeep.sing...@gmail.com> wrote: > > Hi, > > Yes, My client and server both are using certificate and have added in ZK > and client as well. > > Thanks, > --------------------- > Kuldeep Singh Budania > > > > On Thu, Oct 8, 2020 at 5:56 PM Enrico Olivelli <eolive...@gmail.com> wrote: > > > Il giorno gio 8 ott 2020 alle ore 14:17 kuldeep singh < > > kuldeep.sing...@gmail.com> ha scritto: > > > > > Hi Team, > > > > > > I am facing one issue in SSL communication between client and zookeeper > > > server. > > > > > > ZK 3.5.6 version > > > > > > 1. Mi on 3 node > > > 2. Applying SSL and 3 nodes cluster is working fine > > > 3. Scaled my cluster with 2 nodes and now my cluster have 5 nodes over > > SSL > > > > > > but after scaling my SSL is not working between client and ZK server and > > > even not able to login using zkCli as well. > > > > > > Can someone provide the details please why it is happening? > > > > > > > Is your client configured to use SSL ? > > > > Enrico > > > > > > > > > > > > Thanks, > > > --------------------- > > > Kuldeep Singh Budania > > > Software Architect > > > > > > > > > > > > On Mon, Jul 13, 2020 at 2:19 PM Enrico Olivelli - Diennea > > > <enrico.olive...@diennea.com.invalid> wrote: > > > > > > > It looks like we ported it to 3.5. > > > > > > > > See the subtask > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-2792 > > > > > > > > Enrico > > > > > > > > Il giorno 13/07/20, 10:37 "kuldeep singh" <kuldeep.sing...@gmail.com> > > > ha > > > > scritto: > > > > > > > > Hi Team, > > > > > > > > I appreciate it if I will get a response as soon as possible, as I > > am > > > > stuck > > > > at this point. > > > > > > > > Thanks, > > > > --------------------- > > > > Kuldeep Singh Budania > > > > > > > > > > > > > > > > On Mon, Jul 13, 2020 at 11:10 AM kuldeep singh < > > > > kuldeep.sing...@gmail.com> > > > > wrote: > > > > > > > > > Hi Team, > > > > > > > > > > Server to Server communication is not supported in 3.5.6 version > > > as > > > > per > > > > > below JIRA issue? > > > > > > > > > > https://issues.apache.org/jira/browse/ZOOKEEPER-2639 > > > > > > > > > > Thanks, > > > > > --------------------- > > > > > Kuldeep Singh Budania > > > > > > > > > > > > > > > > > > > > On Thu, Jul 2, 2020 at 4:24 PM kuldeep singh < > > > > kuldeep.sing...@gmail.com> > > > > > wrote: > > > > > > > > > >> Thanks for the reply. > > > > >> > > > > >> Now my ZKCli cmd is working fine as we use some our customized > > > > >> authentication and we resolve the issue. > > > > >> > > > > >> Now I am going to implement Server to Server communication. > > > > >> > > > > >> Thanks, > > > > >> --------------------- > > > > >> Kuldeep Singh Budania > > > > >> > > > > >> > > > > >> > > > > >> On Thu, Jul 2, 2020 at 3:53 PM Szalay-Bekő Máté < > > > > >> szalay.beko.m...@gmail.com> wrote: > > > > >> > > > > >>> I think SSL is working for you already... If you managed to > > start > > > > the > > > > >>> zkCli.sh and be able to connect to ZooKeeper on the secure port > > > > and issue > > > > >>> any kind of command (like: " ls / "), then the wire encryption > > is > > > > working > > > > >>> and your server/client communication is secured by ZooKeeper. > > > > >>> > > > > >>> Why you want to run the following command? > > > > >>> addauth ztpasswd zooadmin:<password> > > > > >>> > > > > >>> Do you also want to configure a superDigest user in ZooKeeper? > > > > Please > > > > >>> note > > > > >>> that this command is independent from SSL. If you need to > > create > > > a > > > > >>> username-password pair for digest authentication then please > > use > > > > the > > > > >>> command in the following way: > > > > >>> addauth digest zooadmin:yourSuperSecretPassword > > > > >>> > > > > >>> Kind regards, > > > > >>> Mate > > > > >>> > > > > >>> On Thu, Jul 2, 2020 at 6:59 AM kuldeep singh < > > > > kuldeep.sing...@gmail.com> > > > > >>> wrote: > > > > >>> > > > > >>> > 1. sh zkCli.sh --config /etc/zookeeper -server localhost:2281 > > > > >>> > > > > > >>> > 2. addauth ztpasswd zooadmin:<password> > > > > >>> > > > > > >>> > > > > > >>> > Thanks, > > > > >>> > --------------------- > > > > >>> > Kuldeep Singh Budania > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> > On Thu, Jul 2, 2020 at 9:56 AM kuldeep singh < > > > > >>> kuldeep.sing...@gmail.com> > > > > >>> > wrote: > > > > >>> > > > > > >>> > > Hi Team, > > > > >>> > > > > > > >>> > > Any update on this? > > > > >>> > > > > > > >>> > > Thanks, > > > > >>> > > --------------------- > > > > >>> > > Kuldeep Singh Budania > > > > >>> > > > > > > >>> > > > > > > >>> > > > > > > >>> > > On Wed, Jul 1, 2020 at 6:43 PM kuldeep singh < > > > > >>> kuldeep.sing...@gmail.com> > > > > >>> > > wrote: > > > > >>> > > > > > > >>> > >> Sorry this is my bad, there were server setting like below > > > > >>> > >> > > > > >>> > >> export SERVER_JVMFLAGS=" > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > >>> > >> > > > > >>> > >> > > > > >>> > > > > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > > > >>> > >> -Dzookeeper.ssl.keyStore.password=testpass > > > > >>> > >> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > > > >>> > >> -Dzookeeper.ssl.trustStore.password=testpass" > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> export CLIENT_JVMFLAGS=" > > > > >>> > >> > > > > >>> > >> > > > > >>> > > > > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > > > >>> > >> -Dzookeeper.client.secure=true > > > > >>> > >> > > > > >>> > > > > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > > > >>> > >> -Dzookeeper.ssl.keyStore.password=testpass > > > > >>> > >> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > > > >>> > >> -Dzookeeper.ssl.trustStore.password=testpass" > > > > >>> > >> > > > > >>> > >> I want to have SSL between client to server communication > > > > >>> > >> > > > > >>> > >> I am already following the same link which you have shared > > > > with me > > > > >>> but > > > > >>> > >> that is not working. > > > > >>> > >> > > > > >>> > >> Zoo.cfg > > > > >>> > >> > > > > >>> > >> secureClientPort=2281 > > > > >>> > >> initLimit=10 > > > > >>> > >> syncLimit=5 > > > > >>> > >> dataDir=/var/lib/zookeeper/data > > > > >>> > >> tickTime=2000 > > > > >>> > >> autopurge.snapRetainCount=3 > > > > >>> > >> autopurge.purgeInterval=1 > > > > >>> > >> admin.enableServer=false > > > > >>> > >> standaloneEnabled=false > > > > >>> > >> jute.maxbuffer=2147483648 > > > > >>> > >> > > > > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > >>> > >> server.1=host1_priv:10288:10388 > > > > >>> > >> server.2=host2_priv:10288:10388 > > > > >>> > >> server.3=host3_priv:10288:10388 > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> command to connect using zkcli > > > > >>> > >> > > > > >>> > >> 1. zkcli zoo.cfg localhost:2281 > > > > >>> > >> 2. addauth ztpasswd usernaem:password > > > > >>> > >> > > > > >>> > >> after second step we are getting below error > > > > >>> > >> > > > > >>> > >> WatchedEvent state:AuthFailed type:None path:null > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN > > > > >>> > >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No > > > > authentication > > > > >>> > provider > > > > >>> > >> for scheme: ztpasswd has x509 ip digest* > > > > >>> > >> > > > > >>> > >> Thanks, > > > > >>> > >> --------------------- > > > > >>> > >> Kuldeep Singh Budania > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> > > > > >>> > >> On Wed, Jul 1, 2020 at 6:25 PM Szalay-Bekő Máté < > > > > >>> > >> szalay.beko.m...@gmail.com> wrote: > > > > >>> > >> > > > > >>> > >>> > No authentication provider for scheme: ztpasswd has > > x509 > > > > ip > > > > >>> digest* > > > > >>> > >>> > > > > >>> > >>> This suggest you have some configuration error... Where > > did > > > > you > > > > >>> use the > > > > >>> > >>> "ztpasswd" string in your configs / commands? > > > > >>> > >>> > > > > >>> > >>> On Wed, Jul 1, 2020 at 2:53 PM Szalay-Bekő Máté < > > > > >>> > >>> szalay.beko.m...@gmail.com> > > > > >>> > >>> wrote: > > > > >>> > >>> > > > > >>> > >>> > > My ZK server is up and running in secure mode > > > > >>> > >>> > > > > > >>> > >>> > What is your goal? You want to setup client-server SSL > > > > >>> connection? > > > > >>> > >>> > > > > > >>> > >>> > see: > > > > >>> > >>> > > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide > > > > >>> > >>> > > > > > >>> > >>> > (or you want to have both SSL and SASL enabled?) > > > > >>> > >>> > > > > > >>> > >>> > Anyway, please remove the following line from the > > > > >>> SERVER_JVMFLAGS: > > > > >>> > >>> > > > > > >>> > > > > > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > > > >>> > >>> > This is a configuration that makes sense only for the > > > > ZooKeeper > > > > >>> > client, > > > > >>> > >>> > not for the server. For the server, use the following: > > > > >>> > >>> > > > > > >>> > >>> > > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > >>> > >>> > > > > > >>> > >>> > If it doesn't solve the issue, then can you please send > > > > your > > > > >>> zoo.cfg > > > > >>> > >>> file? > > > > >>> > >>> > Also can you please send the zkCli command you execute? > > > > (you > > > > >>> need to > > > > >>> > >>> > connect to the secure ZooKeeper port, unless > > > > portUnification is > > > > >>> > >>> enabled) > > > > >>> > >>> > > > > > >>> > >>> > Kind regards, > > > > >>> > >>> > Mate > > > > >>> > >>> > > > > > >>> > >>> > On Wed, Jul 1, 2020 at 9:48 AM kuldeep singh < > > > > >>> > >>> kuldeep.sing...@gmail.com> > > > > >>> > >>> > wrote: > > > > >>> > >>> > > > > > >>> > >>> >> Hi, > > > > >>> > >>> >> > > > > >>> > >>> >> we have done below changes in java.env file > > > > >>> > >>> >> > > > > >>> > >>> >> export SERVER_JVMFLAGS=" > > > > >>> > >>> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > > > >>> > >>> >> > > > > >>> > > > > > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > > > >>> > >>> >> -Dzookeeper.ssl.keyStore.password=testpass > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > > > >>> > >>> >> -Dzookeeper.ssl.trustStore.password=testpass" > > > > >>> > >>> >> > > > > >>> > >>> >> > > > > >>> > >>> >> > > > > >>> > >>> >> export CLIENT_JVMFLAGS=" > > > > >>> > >>> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > -Dzookeeper.clientCnxnSocket=org.apache.zookeeper.ClientCnxnSocketNetty > > > > >>> > >>> >> -Dzookeeper.client.secure=true > > > > >>> > >>> >> > > > > >>> > > > > > -Dzookeeper.ssl.keyStore.location=/root/zookeeper/ssl/testKeyStore.jks > > > > >>> > >>> >> -Dzookeeper.ssl.keyStore.password=testpass > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/root/zookeeper/ssl/testTrustStore.jks > > > > >>> > >>> >> -Dzookeeper.ssl.trustStore.password=testpass" > > > > >>> > >>> >> > > > > >>> > >>> >> I have started the ZK server and it is up without any > > > > issue. > > > > >>> > >>> >> > > > > >>> > >>> >> But now when I login to ZkCli then it gives the below > > > > error. > > > > >>> > >>> >> > > > > >>> > >>> >> WatchedEvent state:AuthFailed type:None path:null > > > > >>> > >>> >> > > > > >>> > >>> >> > > > > >>> > >>> >> Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN > > > > >>> > >>> >> [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No > > > > >>> authentication > > > > >>> > >>> provider > > > > >>> > >>> >> for scheme: ztpasswd has x509 ip digest* > > > > >>> > >>> >> > > > > >>> > >>> >> Please help me on this issue > > > > >>> > >>> >> > > > > >>> > >>> >> Thanks, > > > > >>> > >>> >> --------------------- > > > > >>> > >>> >> Kuldeep Singh Budania > > > > >>> > >>> >> Software Architect > > > > >>> > >>> >> > > > > >>> > >>> >> > > > > >>> > >>> >> On Wed, Jul 1, 2020 at 12:05 PM kuldeep singh < > > > > >>> > >>> kuldeep.sing...@gmail.com> > > > > >>> > >>> >> wrote: > > > > >>> > >>> >> > > > > >>> > >>> >> > Hi, > > > > >>> > >>> >> > > > > > >>> > >>> >> > My ZK server is up and running in secure mode, But > > > > When I am > > > > >>> > >>> trying to > > > > >>> > >>> >> > connect to the ZK server using ZKCli, it gives the > > > below > > > > >>> error. > > > > >>> > >>> >> > > > > > >>> > >>> >> > WatchedEvent state:AuthFailed type:None path:null > > > > >>> > >>> >> > > > > > >>> > >>> >> > > > > > >>> > >>> >> > Zookeeper logs :- *2020-07-01 07:38:09,342 - WARN > > > > >>> > >>> >> > [nioEventLoopGroup-4-2:ZooKeeperServer@1119] - No > > > > >>> authentication > > > > >>> > >>> >> provider > > > > >>> > >>> >> > for scheme: ztpasswd has x509 ip digest* > > > > >>> > >>> >> > > > > > >>> > >>> >> > Can someone please help me on this issue. we are > > using > > > > the > > > > >>> 3.5.6 > > > > >>> > >>> >> version. > > > > >>> > >>> >> > > > > > >>> > >>> >> > I appreciate if I will get a response as soon as > > > > possible, as > > > > >>> I am > > > > >>> > >>> stuck > > > > >>> > >>> >> > at this point. > > > > >>> > >>> >> > > > > > >>> > >>> >> > Thanks, > > > > >>> > >>> >> > --------------------- > > > > >>> > >>> >> > Kuldeep Singh Budania > > > > >>> > >>> >> > Software Architect > > > > >>> > >>> >> > > > > > >>> > >>> >> > > > > > >>> > >>> >> > > > > > >>> > >>> >> > On Thu, Jun 25, 2020 at 11:54 AM Enrico Olivelli - > > > > Diennea > > > > >>> > >>> >> > <enrico.olive...@diennea.com.invalid> wrote: > > > > >>> > >>> >> > > > > > >>> > >>> >> >> I mean in zoo.cfg > > > > >>> > >>> >> >> Not as a system property > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Enrico > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Il giorno 25/06/20, 08:19 "Enrico Olivelli - > > > Diennea" < > > > > >>> > >>> >> >> enrico.olive...@diennea.com.INVALID> ha scritto: > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Hi > > > > >>> > >>> >> >> You have to enable Netty on the server side > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Something like: > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > > > > > serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Hope that helps > > > > >>> > >>> >> >> Enrico > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Il giorno 24/06/20, 19:17 "kuldeep singh" < > > > > >>> > >>> >> kuldeep.sing...@gmail.com> > > > > >>> > >>> >> >> ha scritto: > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Hi, > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> I got below error while setting SSL > > > properties > > > > in > > > > >>> > zkEnv.sh > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > ========================================================================================== > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,864 - INFO > > > > >>> > [main:QuorumPeerConfig@133] > > > > >>> > >>> - > > > > >>> > >>> >> >> Reading > > > > >>> > >>> >> >> configuration from: /etc/zookeeper/zoo.cfg > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,874 - INFO > > > > >>> > [main:QuorumPeerConfig@385] > > > > >>> > >>> - > > > > >>> > >>> >> >> clientPortAddress is 0.0.0.0/0.0.0.0:10181 > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,874 - INFO > > > > >>> > [main:QuorumPeerConfig@399] > > > > >>> > >>> - > > > > >>> > >>> >> >> secureClientPortAddress is > > > > 0.0.0.0/0.0.0.0:2281 > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,878 - INFO > > > > [main:X509Util@79] - > > > > >>> > >>> Setting > > > > >>> > >>> >> -D > > > > >>> > >>> >> >> > > > > jdk.tls.rejectClientInitiatedRenegotiation=true to > > > > >>> > disable > > > > >>> > >>> >> >> client-initiated > > > > >>> > >>> >> >> TLS renegotiation > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,897 - INFO > > > > >>> > >>> [main:DatadirCleanupManager@78] > > > > >>> > >>> >> - > > > > >>> > >>> >> >> autopurge.snapRetainCount set to 3 > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,897 - INFO > > > > >>> > >>> [main:DatadirCleanupManager@79] > > > > >>> > >>> >> - > > > > >>> > >>> >> >> autopurge.purgeInterval set to 1 > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,898 - INFO [ > > > > >>> > >>> >> >> > > PurgeTask:DatadirCleanupManager$PurgeTask@138 > > > ] > > > > - > > > > >>> Purge > > > > >>> > >>> task > > > > >>> > >>> >> >> started. > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,899 - INFO > > > > [main:ManagedUtil@46] > > > > >>> - > > > > >>> > >>> Log4j > > > > >>> > >>> >> >> found with > > > > >>> > >>> >> >> jmx enabled. > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,903 - INFO > > > > >>> > >>> [PurgeTask:FileTxnSnapLog@103] > > > > >>> > >>> >> - > > > > >>> > >>> >> >> zookeeper.snapshot.trust.empty : false > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,910 - INFO [ > > > > >>> > >>> >> >> > > PurgeTask:DatadirCleanupManager$PurgeTask@144 > > > ] > > > > - > > > > >>> Purge > > > > >>> > >>> task > > > > >>> > >>> >> >> completed. > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,975 - INFO > > > > >>> [main:QuorumPeerMain@141 > > > > >>> > ] > > > > >>> > >>> - > > > > >>> > >>> >> >> Starting quorum > > > > >>> > >>> >> >> peer > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,983 - INFO > > > > >>> > [main:ServerCnxnFactory@135] > > > > >>> > >>> - > > > > >>> > >>> >> >> Using > > > > >>> > >>> >> >> > > > > org.apache.zookeeper.server.NIOServerCnxnFactory as > > > > >>> > server > > > > >>> > >>> >> >> connection > > > > >>> > >>> >> >> factory > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,986 - INFO > > > > >>> > >>> [main:NIOServerCnxnFactory@673] > > > > >>> > >>> >> - > > > > >>> > >>> >> >> Configuring NIO connection handler with 10s > > > > >>> sessionless > > > > >>> > >>> >> >> connection timeout, > > > > >>> > >>> >> >> 2 selector thread(s), 16 worker threads, > > and > > > > 64 kB > > > > >>> direct > > > > >>> > >>> >> buffers. > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,992 - INFO > > > > >>> > >>> [main:NIOServerCnxnFactory@686 > > > > >>> > >>> >> ] > > > > >>> > >>> >> >> - binding > > > > >>> > >>> >> >> to port 0.0.0.0/0.0.0.0:10181 > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,994 - INFO > > > > >>> > [main:ServerCnxnFactory@135] > > > > >>> > >>> - > > > > >>> > >>> >> >> Using > > > > >>> > >>> >> >> > > > > org.apache.zookeeper.server.NIOServerCnxnFactory as > > > > >>> > server > > > > >>> > >>> >> >> connection > > > > >>> > >>> >> >> factory > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> 2020-06-24 15:49:35,995 - ERROR > > > > >>> [main:QuorumPeerMain@101 > > > > >>> > ] > > > > >>> > >>> - > > > > >>> > >>> >> >> Unexpected > > > > >>> > >>> >> >> exception, exiting abnormally > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> java.lang.UnsupportedOperationException: > > SSL > > > > isn't > > > > >>> > >>> supported in > > > > >>> > >>> >> >> NIOServerCnxn > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> at > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:644) > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> at > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:155) > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> at > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:123) > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> at > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82) > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> I have set the following properties in > > > > >>> SERVER_JVMFLAGS in > > > > >>> > >>> >> >> zkEnv.sh file : > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > "-Dzookeeper.serverCnxnFactory=org.apache.zookeeper.server.NettyServerCnxnFactory > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.ssl.keyStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkkeystore.p12 > > > > >>> > >>> >> >> > > > > -Dzookeeper.ssl.keyStore.password=EvaiKiO1@123456 > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > -Dzookeeper.ssl.trustStore.location=/var/opt/vs/SecureInterface/keystore/CassSpkTrustStore.jks > > > > >>> > >>> >> >> > > > > -Dzookeeper.ssl.trustStore.password=EvaiKiO1@123456" > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> Thanks, > > > > >>> > >>> >> >> --------------------- > > > > >>> > >>> >> >> Kuldeep Singh Budania > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> On Mon, Jun 22, 2020 at 8:08 PM Jordan > > > > Zimmerman < > > > > >>> > >>> >> >> jor...@jordanzimmerman.com> > > > > >>> > >>> >> >> wrote: > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > It's the same as the normal ZooKeeper > > > client: > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions > > > > >>> > >>> >> >> > < > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > https://zookeeper.apache.org/doc/r3.6.1/zookeeperAdmin.html#sc_authOptions > > > > >>> > >>> >> >> > > > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > -Jordan > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > On Jun 22, 2020, at 5:50 AM, kuldeep > > > singh > > > > < > > > > >>> > >>> >> >> kuldeep.sing...@gmail.com> > > > > >>> > >>> >> >> > wrote: > > > > >>> > >>> >> >> > > > > > > >>> > >>> >> >> > > Hi Team, > > > > >>> > >>> >> >> > > How we will do secure communication > > > > between the > > > > >>> > Curator > > > > >>> > >>> >> >> framework and > > > > >>> > >>> >> >> > > zookeeper 3.5.6 ? > > > > >>> > >>> >> >> > > I didn't get any solution right now. > > > > >>> > >>> >> >> > > I appreciate it if someone could help > > me > > > > with the > > > > >>> > same. > > > > >>> > >>> >> >> > > > > > > >>> > >>> >> >> > > Thanks, > > > > >>> > >>> >> >> > > --------------------- > > > > >>> > >>> >> >> > > Kuldeep Singh Budania > > > > >>> > >>> >> >> > > Software Architect > > > > >>> > >>> >> >> > > > > > > >>> > >>> >> >> > > > > > > >>> > >>> >> >> > > On Fri, Apr 17, 2020 at 4:53 PM > > > > Szalay-Bekő Máté > > > > >>> < > > > > >>> > >>> >> >> > szalay.beko.m...@gmail.com> > > > > >>> > >>> >> >> > > wrote: > > > > >>> > >>> >> >> > > > > > > >>> > >>> >> >> > >> Hello Kuldeep, > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> did you download the source from the > > > > ZooKeeper > > > > >>> > >>> webpage, or > > > > >>> > >>> >> >> checked out > > > > >>> > >>> >> >> > from > > > > >>> > >>> >> >> > >> git? > > > > >>> > >>> >> >> > >> Anyway, the following commands should > > > > work: > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> wget > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > https://downloads.apache.org/zookeeper/zookeeper-3.5.7/apache-zookeeper-3.5.7.tar.gz > > > > >>> > >>> >> >> > >> tar xzvf > > ./apache-zookeeper-3.5.7.tar.gz > > > > >>> > >>> >> >> > >> cd apache-zookeeper-3.5.7 > > > > >>> > >>> >> >> > >> mvn clean install -DskipTests > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> I tested it with OpenJDK 8u424 and > > maven > > > > 3.6.0. > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> Kind regards, > > > > >>> > >>> >> >> > >> Mate > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >> On Fri, Apr 17, 2020 at 12:51 PM > > kuldeep > > > > singh < > > > > >>> > >>> >> >> > kuldeep.sing...@gmail.com> > > > > >>> > >>> >> >> > >> wrote: > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > >>> Hi All, > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> I have download the zookeeper source > > > > code and > > > > >>> run > > > > >>> > >>> the mvn > > > > >>> > >>> >> >> command for > > > > >>> > >>> >> >> > >>> packaging but get problem on > > > > "zookeeper-server" > > > > >>> > >>> >> directory. > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> *command :- mvn clean package* > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> [ERROR] Failed to execute goal > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> > > > > org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile > > > > >>> > >>> >> >> > >>> (default-compile) on project > > zookeeper: > > > > Fatal > > > > >>> error > > > > >>> > >>> >> >> compiling: > > > > >>> > >>> >> >> > >>> java.lang.NullPointerException -> > > [Help > > > > 1] > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> org.apache.maven.lifecycle.LifecycleExecutionException: > > > > >>> > >>> >> >> Failed to > > > > >>> > >>> >> >> > execute > > > > >>> > >>> >> >> > >>> goal > > > > >>> > >>> >> >> > > > > org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile > > > > >>> > >>> >> >> > >>> (default-compile) on project > > zookeeper: > > > > Fatal > > > > >>> error > > > > >>> > >>> >> >> compiling > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:108) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:76) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:116) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > > >>> > >>> >> > > > > org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:361) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > org.apache.maven.DefaultMaven.execute(DefaultMaven.java:155) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > org.apache.maven.cli.MavenCli.execute(MavenCli.java:584) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > org.apache.maven.cli.MavenCli.doMain(MavenCli.java:213) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > org.apache.maven.cli.MavenCli.main(MavenCli.java:157) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native > > > > Method) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> java.lang.reflect.Method.invoke(Method.java:498) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356) > > > > >>> > >>> >> >> > >>> Caused by: > > > > >>> > >>> >> org.apache.maven.plugin.MojoExecutionException: > > > > >>> > >>> >> >> Fatal error > > > > >>> > >>> >> >> > >>> compiling > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1140) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:188) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208) > > > > >>> > >>> >> >> > >>> ... 19 more > > > > >>> > >>> >> >> > >>> Caused by: > > > > >>> > >>> >> org.codehaus.plexus.compiler.CompilerException: > > > > >>> > >>> >> >> > >>> java.lang.NullPointerException > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:173) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.codehaus.plexus.compiler.javac.JavacCompiler.performCompile(JavacCompiler.java:174) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:1129) > > > > >>> > >>> >> >> > >>> ... 22 more > > > > >>> > >>> >> >> > >>> Caused by: > > java.lang.RuntimeException: > > > > >>> > >>> >> >> java.lang.NullPointerException > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > com.sun.tools.javac.main.Main.compile(Main.java:559) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:129) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:138) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.codehaus.plexus.compiler.javac.JavaxToolsCompiler.compileInProcess(JavaxToolsCompiler.java:126) > > > > >>> > >>> >> >> > >>> ... 24 more > > > > >>> > >>> >> >> > >>> Caused by: > > > java.lang.NullPointerException > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > com.sun.tools.javac.model.JavacTypes.getOverriddenMethods(JavacTypes.java:306) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> com.sun.tools.doclint.Env.setCurrent(Env.java:155) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > com.sun.tools.doclint.Checker.scan(Checker.java:144) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > com.sun.tools.doclint.DocLint$2.visitDecl(DocLint.java:285) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:359) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.visitNewClass(TreeScanner.java:280) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.tools.javac.tree.JCTree$JCNewClass.accept(JCTree.java:1532) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.visitVariable(TreeScanner.java:153) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:373) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitVariable(DocLint.java:346) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.javac.tree.JCTree$JCVariableDecl.accept(JCTree.java:864) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:81) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > >>> > > com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:133) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:360) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitClass(DocLint.java:346) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:720) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > com.sun.source.util.TreeScanner.scan(TreeScanner.java:91) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:99) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > com.sun.source.util.TreeScanner.visitCompilationUnit(TreeScanner.java:120) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:354) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > com.sun.tools.doclint.DocLint$DeclScanner.visitCompilationUnit(DocLint.java:346) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:550) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:68) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > > > com.sun.tools.doclint.DocLint$3.started(DocLint.java:296) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.started(ClientCodeWrapper.java:668) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > com.sun.tools.javac.api.MultiTaskListener.started(MultiTaskListener.java:103) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.javac.main.JavaCompiler.attribute(JavaCompiler.java:1240) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.javac.main.JavaCompiler.compile2(JavaCompiler.java:901) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:860) > > > > >>> > >>> >> >> > >>> at > > > > >>> > >>> >> >> > > com.sun.tools.javac.main.Main.compile(Main.java:523) > > > > >>> > >>> >> >> > >>> ... 27 more > > > > >>> > >>> >> >> > >>> [ERROR] > > > > >>> > >>> >> >> > >>> [ERROR] Re-run Maven using the -X > > > switch > > > > to > > > > >>> enable > > > > >>> > >>> full > > > > >>> > >>> >> >> debug logging. > > > > >>> > >>> >> >> > >>> [ERROR] > > > > >>> > >>> >> >> > >>> [ERROR] For more information about > > the > > > > errors > > > > >>> and > > > > >>> > >>> >> possible > > > > >>> > >>> >> >> solutions, > > > > >>> > >>> >> >> > >>> please read the following articles: > > > > >>> > >>> >> >> > >>> [ERROR] [Help 1] > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> Can someone help me on this issue? > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> Thanks, > > > > >>> > >>> >> >> > >>> --------------------- > > > > >>> > >>> >> >> > >>> Kuldeep Singh Budania > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>> On Sat, Apr 4, 2020 at 5:57 PM > > > > Szalay-Bekő > > > > >>> Máté < > > > > >>> > >>> >> >> > >>> szalay.beko.m...@gmail.com> > > > > >>> > >>> >> >> > >>> wrote: > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >>>> these exceptions can mean many > > > > things... I > > > > >>> think > > > > >>> > >>> this > > > > >>> > >>> >> can > > > > >>> > >>> >> >> be even > > > > >>> > >>> >> >> > >> normal > > > > >>> > >>> >> >> > >>>> duding rolling restart (as some > > > > connections > > > > >>> get > > > > >>> > >>> broken > > > > >>> > >>> >> in > > > > >>> > >>> >> >> this case) > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> However, I saw cases already when > > > > exceptions > > > > >>> like > > > > >>> > >>> these > > > > >>> > >>> >> >> killed > > > > >>> > >>> >> >> > receiver > > > > >>> > >>> >> >> > >>> or > > > > >>> > >>> >> >> > >>>> sender threads in QuorumCnxManager / > > > > Leader > > > > >>> > >>> Election in > > > > >>> > >>> >> >> such a way > > > > >>> > >>> >> >> > that > > > > >>> > >>> >> >> > >>>> they were not able to recover, so > > the > > > > node was > > > > >>> > >>> unable to > > > > >>> > >>> >> >> connect to > > > > >>> > >>> >> >> > any > > > > >>> > >>> >> >> > >>>> quorum until restart. I remember > > > seeing > > > > this > > > > >>> in > > > > >>> > 3.4 > > > > >>> > >>> too. > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> Do you see these exceptions in the > > > > second > > > > >>> server > > > > >>> > >>> (the > > > > >>> > >>> >> one > > > > >>> > >>> >> >> which you > > > > >>> > >>> >> >> > >> just > > > > >>> > >>> >> >> > >>>> upgraded in step 3)? > > > > >>> > >>> >> >> > >>>> Is this issue reproducible? > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> What is the tickTime and initLimit > > you > > > > use? > > > > >>> Maybe > > > > >>> > >>> the > > > > >>> > >>> >> >> server just > > > > >>> > >>> >> >> > >> require > > > > >>> > >>> >> >> > >>>> more time to sync? > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> I would need more logs to really see > > > > what > > > > >>> > happened. > > > > >>> > >>> Can > > > > >>> > >>> >> >> you create a > > > > >>> > >>> >> >> > >> Jira > > > > >>> > >>> >> >> > >>>> issue and upload the logs and also > > the > > > > >>> ZooKeeper > > > > >>> > >>> >> configs? > > > > >>> > >>> >> >> I am happy > > > > >>> > >>> >> >> > to > > > > >>> > >>> >> >> > >>>> take a closer look. > > > > >>> > >>> >> >> > >>>> (if you need to re-run the test to > > > > collect the > > > > >>> > logs, > > > > >>> > >>> >> then > > > > >>> > >>> >> >> enabling > > > > >>> > >>> >> >> > >> DEBUG > > > > >>> > >>> >> >> > >>>> logs would be great. The INFO level > > > > logs are > > > > >>> > usually > > > > >>> > >>> >> >> enough for these > > > > >>> > >>> >> >> > >>>> problems, but one can never know...) > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> Kind regards, > > > > >>> > >>> >> >> > >>>> Mate > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> On Fri, Apr 3, 2020 at 10:05 AM > > > kuldeep > > > > singh > > > > >>> < > > > > >>> > >>> >> >> > >> kuldeep.sing...@gmail.com > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>> wrote: > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>>>> Hi Team, > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> I have done some POC on rolling > > > > upgrade and > > > > >>> found > > > > >>> > >>> below > > > > >>> > >>> >> >> result. > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> 1. On 1st node upgrade zookeeper > > . > > > > Traffic > > > > >>> was > > > > >>> > >>> >> running > > > > >>> > >>> >> >> fine > > > > >>> > >>> >> >> > >> because > > > > >>> > >>> >> >> > >>> 2 > > > > >>> > >>> >> >> > >>>>> nodes are already on old > > zookeeper. > > > > >>> > >>> >> >> > >>>>> 2. On 1st node upgrade our > > > > application and > > > > >>> > didn’t > > > > >>> > >>> >> find > > > > >>> > >>> >> >> any issue > > > > >>> > >>> >> >> > >>>>> 3. On 2nd node upgrade zookeeper > > > but > > > > got > > > > >>> below > > > > >>> > >>> error > > > > >>> > >>> >> >> and zookeeper > > > > >>> > >>> >> >> > >>> is > > > > >>> > >>> >> >> > >>>>> not taking any requests > > > > >>> > >>> >> >> > >>>>> 4. > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> java.io.EOFException > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > > > > java.io.DataInputStream.readInt(DataInputStream.java:392) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:747) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> 2020-03-30 14:19:55,587 - WARN > > > > >>> > >>> >> >> > >>>>> > > > > [RecvWorker:1:QuorumCnxManager$RecvWorker@765] > > > > >>> - > > > > >>> > >>> >> >> Interrupting > > > > >>> > >>> >> >> > >>> SendWorker > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> 2020-03-30 14:19:55,588 - ERROR > > > > >>> [LearnerHandler-/ > > > > >>> > >>> >> >> 192.168.44.73:33754 > > > > >>> > >>> >> >> > >>>>> :LearnerHandler@562] - Unexpected > > > > exception > > > > >>> > >>> causing > > > > >>> > >>> >> >> shutdown while > > > > >>> > >>> >> >> > >>> sock > > > > >>> > >>> >> >> > >>>>> still open > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> java.io.EOFException > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > > > > java.io.DataInputStream.readInt(DataInputStream.java:392) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > >>> > >>> > > > > >>> > > > > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.quorum.QuorumPacket.deserialize(QuorumPacket.java:83) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> at > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > >>> > >>> > > > > >>> > > > > > >>> > > > > > > > > > org.apache.zookeeper.server.quorum.LearnerHandler.run(LearnerHandler.java:476) > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> 2020-03-30 14:19:55,588 - WARN > > > > >>> > >>> >> >> > >>>>> > > > > [SendWorker:1:QuorumCnxManager$SendWorker@679] > > > > >>> - > > > > >>> > >>> >> >> Interrupted while > > > > >>> > >>> >> >> > >>>> waiting > > > > >>> > >>> >> >> > >>>>> for message on queue > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> Please let me know is this the > > known > > > > issue or > > > > >>> > this > > > > >>> > >>> is > > > > >>> > >>> >> >> different issue > > > > >>> > >>> >> >> > >>>> which > > > > >>> > >>> >> >> > >>>>> is mention in Apache zookeeper > > > > documentation > > > > >>> when > > > > >>> > >>> >> >> upgrading from > > > > >>> > >>> >> >> > >> 3.4.5 > > > > >>> > >>> >> >> > >>> to > > > > >>> > >>> >> >> > >>>>> 3.5.6 > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> Thanks, > > > > >>> > >>> >> >> > >>>>> --------------------- > > > > >>> > >>> >> >> > >>>>> Kuldeep Singh Budania > > > > >>> > >>> >> >> > >>>>> Software Architect > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>> On Sun, Mar 29, 2020 at 9:06 AM > > > > Alexander > > > > >>> Shraer > > > > >>> > < > > > > >>> > >>> >> >> shra...@gmail.com> > > > > >>> > >>> >> >> > >>>>> wrote: > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>>>> +1 to what Mate said (I wrote the > > > > quoted > > > > >>> > >>> >> instructions). > > > > >>> > >>> >> >> > >>>>>> > > > > >>> > >>> >> >> > >>>>>> > > > > >>> > >>> >> >> > >>>>>> > > > > >>> > >>> >> >> > >>>>>> On Tue, Mar 24, 2020 at 7:03 AM > > > > Szalay-Bekő > > > > >>> > Máté < > > > > >>> > >>> >> >> > >>>>>> szalay.beko.m...@gmail.com> > > > > >>> > >>> >> >> > >>>>>> wrote: > > > > >>> > >>> >> >> > >>>>>> > > > > >>> > >>> >> >> > >>>>>>> Hi Kuldeep, > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> I just want to provide you some > > > > background > > > > >>> info > > > > >>> > >>> about > > > > >>> > >>> >> >> our > > > > >>> > >>> >> >> > >>>>> documentation. > > > > >>> > >>> >> >> > >>>>>>> The reason to upgrade to 3.4.6 > > > first > > > > is to > > > > >>> > avoid > > > > >>> > >>> the > > > > >>> > >>> >> >> following > > > > >>> > >>> >> >> > >>> error: > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> 2013-01-30 11:32:10,663 > > [myid:2] - > > > > WARN > > > > >>> > >>> [localhost/ > > > > >>> > >>> >> >> > >>> 127.0.0.1:2784 > > > > >>> > >>> >> >> > >>>>>>> :QuorumCnxManager@349] - Invalid > > > > server > > > > >>> id: > > > > >>> > >>> -65536 > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> This error comes because of the > > > > protocol > > > > >>> > changes > > > > >>> > >>> >> between > > > > >>> > >>> >> >> > >> ZooKeeper > > > > >>> > >>> >> >> > >>>>> server > > > > >>> > >>> >> >> > >>>>>>> nodes during connection > > initiation > > > > for > > > > >>> leader > > > > >>> > >>> >> election. > > > > >>> > >>> >> >> In > > > > >>> > >>> >> >> > >>> ZooKeeper > > > > >>> > >>> >> >> > >>>>> 3.5 > > > > >>> > >>> >> >> > >>>>>> a > > > > >>> > >>> >> >> > >>>>>>> protocol version was introduced > > > (see > > > > >>> > >>> ZOOKEEPER-107) > > > > >>> > >>> >> and > > > > >>> > >>> >> >> since > > > > >>> > >>> >> >> > >> that > > > > >>> > >>> >> >> > >>>> time > > > > >>> > >>> >> >> > >>>>>> the > > > > >>> > >>> >> >> > >>>>>>> fist long value sent in the > > initial > > > > >>> message is > > > > >>> > >>> not > > > > >>> > >>> >> the > > > > >>> > >>> >> >> server ID > > > > >>> > >>> >> >> > >>> but > > > > >>> > >>> >> >> > >>>>> the > > > > >>> > >>> >> >> > >>>>>>> protocol version (-65536). In > > > > ZooKeeper > > > > >>> 3.4.6 > > > > >>> > we > > > > >>> > >>> made > > > > >>> > >>> >> >> the old 3.4 > > > > >>> > >>> >> >> > >>>>>>> ZooKeepers backward compatible, > > so > > > > they are > > > > >>> > able > > > > >>> > >>> to > > > > >>> > >>> >> >> parse both > > > > >>> > >>> >> >> > >> the > > > > >>> > >>> >> >> > >>>> old > > > > >>> > >>> >> >> > >>>>>> and > > > > >>> > >>> >> >> > >>>>>>> the new protocol format (see > > > > >>> ZOOKEEPER-1633). > > > > >>> > >>> This > > > > >>> > >>> >> >> issue happens > > > > >>> > >>> >> >> > >>> only > > > > >>> > >>> >> >> > >>>>>> when > > > > >>> > >>> >> >> > >>>>>>> you need to use old (3.4.0 - > > 3.4.5) > > > > and new > > > > >>> > >>> (3.5.0+) > > > > >>> > >>> >> >> ZooKeeper > > > > >>> > >>> >> >> > >>>> servers > > > > >>> > >>> >> >> > >>>>>>> together in the same cluster. > > > During > > > > a > > > > >>> rolling > > > > >>> > >>> >> upgrade, > > > > >>> > >>> >> >> this is > > > > >>> > >>> >> >> > >>>> usually > > > > >>> > >>> >> >> > >>>>>> the > > > > >>> > >>> >> >> > >>>>>>> case to have old and new > > ZooKeepers > > > > present > > > > >>> > >>> together. > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> The fact that you haven't seen > > any > > > > issues > > > > >>> might > > > > >>> > >>> be > > > > >>> > >>> >> >> caused by the > > > > >>> > >>> >> >> > >>>> order > > > > >>> > >>> >> >> > >>>>> of > > > > >>> > >>> >> >> > >>>>>>> the servers. In ZooKeeper the > > > > connection > > > > >>> > >>> initiation > > > > >>> > >>> >> >> between the > > > > >>> > >>> >> >> > >>>> servers > > > > >>> > >>> >> >> > >>>>>>> during the leader election > > follows > > > a > > > > >>> specific > > > > >>> > >>> rule. > > > > >>> > >>> >> As > > > > >>> > >>> >> >> far as I > > > > >>> > >>> >> >> > >>>>> remember > > > > >>> > >>> >> >> > >>>>>>> always the server with the larger > > > ID > > > > 'wins > > > > >>> the > > > > >>> > >>> >> >> challenge', so it > > > > >>> > >>> >> >> > >> is > > > > >>> > >>> >> >> > >>>>>>> possible, that the old server > > > didn't > > > > need > > > > >>> to > > > > >>> > >>> parse > > > > >>> > >>> >> any > > > > >>> > >>> >> >> initial > > > > >>> > >>> >> >> > >>>> message > > > > >>> > >>> >> >> > >>>>>> (if > > > > >>> > >>> >> >> > >>>>>>> it had the largest ID) and this > > is > > > > why you > > > > >>> > >>> haven't > > > > >>> > >>> >> seen > > > > >>> > >>> >> >> the > > > > >>> > >>> >> >> > >> issue. > > > > >>> > >>> >> >> > >>>> Also > > > > >>> > >>> >> >> > >>>>>>> having 2 nodes up from the 3 > > nodes > > > > cluster > > > > >>> > still > > > > >>> > >>> >> makes > > > > >>> > >>> >> >> the > > > > >>> > >>> >> >> > >> cluster > > > > >>> > >>> >> >> > >>>> work > > > > >>> > >>> >> >> > >>>>>> (so > > > > >>> > >>> >> >> > >>>>>>> you should also check if all the > > > > servers > > > > >>> are > > > > >>> > >>> part of > > > > >>> > >>> >> >> the quorum). > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> I agree with Enrico and Norbert, > > > the > > > > >>> safest and > > > > >>> > >>> most > > > > >>> > >>> >> >> stable way > > > > >>> > >>> >> >> > >> is > > > > >>> > >>> >> >> > >>>>>> upgrade > > > > >>> > >>> >> >> > >>>>>>> first to 3.4.latest, then go to > > > > 3.5.latest. > > > > >>> > >>> Still, if > > > > >>> > >>> >> >> you don't > > > > >>> > >>> >> >> > >> see > > > > >>> > >>> >> >> > >>>>> that > > > > >>> > >>> >> >> > >>>>>>> you would hit this specific issue > > > > (e.g. no > > > > >>> > >>> "Invalid > > > > >>> > >>> >> >> server id" in > > > > >>> > >>> >> >> > >>> the > > > > >>> > >>> >> >> > >>>>> log > > > > >>> > >>> >> >> > >>>>>>> files), and all the three servers > > > can > > > > >>> handle > > > > >>> > >>> traffic, > > > > >>> > >>> >> >> then maybe > > > > >>> > >>> >> >> > >>> you > > > > >>> > >>> >> >> > >>>>>> don't > > > > >>> > >>> >> >> > >>>>>>> need to upgrade first to > > > 3.4.latest, > > > > it is > > > > >>> your > > > > >>> > >>> >> >> decision. > > > > >>> > >>> >> >> > >>> Definitely > > > > >>> > >>> >> >> > >>>>> you > > > > >>> > >>> >> >> > >>>>>>> should test it first, as > > suggested > > > > by the > > > > >>> > others. > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> Kind regards, > > > > >>> > >>> >> >> > >>>>>>> Mate > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> On Tue, Mar 24, 2020 at 12:29 PM > > > > Norbert > > > > >>> Kalmar > > > > >>> > >>> >> >> > >>>>>>> <nkal...@cloudera.com.invalid> > > > > wrote: > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> Hi, > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> That guide is to upgrade to > > 3.5.0, > > > > which > > > > >>> was > > > > >>> > an > > > > >>> > >>> >> alpha > > > > >>> > >>> >> >> version. > > > > >>> > >>> >> >> > >> A > > > > >>> > >>> >> >> > >>>> lot > > > > >>> > >>> >> >> > >>>>>> has > > > > >>> > >>> >> >> > >>>>>>>> changed for the first stable > > > > release of > > > > >>> 3.5.5 > > > > >>> > >>> and > > > > >>> > >>> >> then > > > > >>> > >>> >> >> a few > > > > >>> > >>> >> >> > >>> more, > > > > >>> > >>> >> >> > >>>>> even > > > > >>> > >>> >> >> > >>>>>>>> rolling upgrade issues have been > > > > fixed for > > > > >>> > >>> 3.5.6. > > > > >>> > >>> >> >> > >>>>>>>> This is a more up-to-date guide: > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > > > > >>> > > > > > https://cwiki.apache.org/confluence/display/ZOOKEEPER/Upgrade+FAQ > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> If you have done your testing > > > (with > > > > prod > > > > >>> > >>> snapshot!), > > > > >>> > >>> >> >> then you > > > > >>> > >>> >> >> > >> can > > > > >>> > >>> >> >> > >>>>> skip > > > > >>> > >>> >> >> > >>>>>>> 3.4 > > > > >>> > >>> >> >> > >>>>>>>> latest upgrade, but keep in mind > > > we > > > > do our > > > > >>> > >>> >> >> recommendations for > > > > >>> > >>> >> >> > >> a > > > > >>> > >>> >> >> > >>>>>> reason. > > > > >>> > >>> >> >> > >>>>>>>> There were issues reported > > and/or > > > > found > > > > >>> during > > > > >>> > >>> >> >> testing. Some > > > > >>> > >>> >> >> > >> are > > > > >>> > >>> >> >> > >>>>> fixed > > > > >>> > >>> >> >> > >>>>>>> with > > > > >>> > >>> >> >> > >>>>>>>> 3.5.6, some only happens if > > > certain > > > > >>> conditions > > > > >>> > >>> stand > > > > >>> > >>> >> >> > >>> (IOException: > > > > >>> > >>> >> >> > >>>> No > > > > >>> > >>> >> >> > >>>>>>>> snapshot found - mentioned in > > the > > > > guide, > > > > >>> fixed > > > > >>> > >>> in > > > > >>> > >>> >> >> 3.5.6). > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> So it is up to you, I would > > still > > > > >>> recommend to > > > > >>> > >>> do an > > > > >>> > >>> >> >> 3.4 > > > > >>> > >>> >> >> > >> upgrade > > > > >>> > >>> >> >> > >>>>> first, > > > > >>> > >>> >> >> > >>>>>>> if > > > > >>> > >>> >> >> > >>>>>>>> it's feasible. > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> Regards, > > > > >>> > >>> >> >> > >>>>>>>> Norbert > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> On Tue, Mar 24, 2020 at 11:45 AM > > > > kuldeep > > > > >>> > singh < > > > > >>> > >>> >> >> > >>>>>>> kuldeep.sing...@gmail.com> > > > > >>> > >>> >> >> > >>>>>>>> wrote: > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> Hi, > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> Current Zookeeper version :- > > > 3.4.5 > > > > >>> > >>> >> >> > >>>>>>>>> Upgraded version > > > :- > > > > 3.5.6 > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> We are not going with 3.5.7. > > Our > > > > final > > > > >>> > >>> decision is > > > > >>> > >>> >> >> zookeeper > > > > >>> > >>> >> >> > >>>>> version > > > > >>> > >>> >> >> > >>>>>> is > > > > >>> > >>> >> >> > >>>>>>>>> 3.5.6 > > > > >>> > >>> >> >> > >>>>>>>>> as per your reply first we need > > > to > > > > move > > > > >>> > latest > > > > >>> > >>> >> >> version of > > > > >>> > >>> >> >> > >>> 3.4.x, > > > > >>> > >>> >> >> > >>>>> like > > > > >>> > >>> >> >> > >>>>>>>> below > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> 3.4.5 -> 3.4.14 -> 3.5.6 > > (Correct > > > > me if > > > > >>> I am > > > > >>> > >>> wrong > > > > >>> > >>> >> >> here) > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> But if We are not facing any > > > > problem > > > > >>> that i > > > > >>> > >>> have > > > > >>> > >>> >> >> shared you > > > > >>> > >>> >> >> > >>> that > > > > >>> > >>> >> >> > >>>> we > > > > >>> > >>> >> >> > >>>>>>> have > > > > >>> > >>> >> >> > >>>>>>>>> set up of 3 node cluster where > > 2 > > > > node > > > > >>> are on > > > > >>> > >>> 3.5.6 > > > > >>> > >>> >> >> version > > > > >>> > >>> >> >> > >> and > > > > >>> > >>> >> >> > >>> 1 > > > > >>> > >>> >> >> > >>>>> node > > > > >>> > >>> >> >> > >>>>>>> on > > > > >>> > >>> >> >> > >>>>>>>>> 3.4.5, Everything is running > > fine > > > > and > > > > >>> didn't > > > > >>> > >>> get > > > > >>> > >>> >> any > > > > >>> > >>> >> >> issue, > > > > >>> > >>> >> >> > >> So > > > > >>> > >>> >> >> > >>>> what > > > > >>> > >>> >> >> > >>>>>>> other > > > > >>> > >>> >> >> > >>>>>>>>> problem we can face if we > > > directly > > > > move > > > > >>> to > > > > >>> > >>> 3.5.6 > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> Thanks, > > > > >>> > >>> >> >> > >>>>>>>>> --------------------- > > > > >>> > >>> >> >> > >>>>>>>>> Kuldeep Singh Budania > > > > >>> > >>> >> >> > >>>>>>>>> Software Architect > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> On Tue, Mar 24, 2020 at 3:58 PM > > > > Enrico > > > > >>> > >>> Olivelli < > > > > >>> > >>> >> >> > >>>>> eolive...@gmail.com > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> wrote: > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>> Hi > > > > >>> > >>> >> >> > >>>>>>>>>> You have to upgrade to latest > > > > 3.4.x > > > > >>> > Zookeeper > > > > >>> > >>> then > > > > >>> > >>> >> >> you will > > > > >>> > >>> >> >> > >>>>> upgrade > > > > >>> > >>> >> >> > >>>>>>> to > > > > >>> > >>> >> >> > >>>>>>>>>> 3.5.7. > > > > >>> > >>> >> >> > >>>>>>>>>> All should run well without > > > issues > > > > >>> > >>> >> >> > >>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>> Enrico > > > > >>> > >>> >> >> > >>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>> Il Mar 24 Mar 2020, 10:18 > > > kuldeep > > > > singh > > > > >>> < > > > > >>> > >>> >> >> > >>>>> kuldeep.sing...@gmail.com > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>>> ha > > > > >>> > >>> >> >> > >>>>>>>>>> scritto: > > > > >>> > >>> >> >> > >>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> Hi Team, > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> We are upgrading zookeeper > > from > > > > 3.4.5 > > > > >>> to > > > > >>> > >>> 3.5.6. I > > > > >>> > >>> >> >> have > > > > >>> > >>> >> >> > >> set > > > > >>> > >>> >> >> > >>>> up 3 > > > > >>> > >>> >> >> > >>>>>>> node > > > > >>> > >>> >> >> > >>>>>>>>>>> cluster where 2 node are on > > > 3.5.6 > > > > >>> version > > > > >>> > >>> and 1 > > > > >>> > >>> >> >> node on > > > > >>> > >>> >> >> > >>>> 3.4.5. > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> Everything is running fine > > and > > > > didn't > > > > >>> get > > > > >>> > any > > > > >>> > >>> >> issue > > > > >>> > >>> >> >> on my > > > > >>> > >>> >> >> > >>>>> system. > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> but I found something on > > apache > > > > site > > > > >>> that > > > > >>> > >>> first > > > > >>> > >>> >> we > > > > >>> > >>> >> >> need > > > > >>> > >>> >> >> > >> to > > > > >>> > >>> >> >> > >>>>>> upgrade > > > > >>> > >>> >> >> > >>>>>>>> on > > > > >>> > >>> >> >> > >>>>>>>>>>> 3.4.6 than we can upgrade to > > > > 3.5.6. So > > > > >>> is > > > > >>> > it > > > > >>> > >>> >> >> mandatory > > > > >>> > >>> >> >> > >> to > > > > >>> > >>> >> >> > >>> go > > > > >>> > >>> >> >> > >>>>> on > > > > >>> > >>> >> >> > >>>>>>>> 3.4.6 > > > > >>> > >>> >> >> > >>>>>>>>>>> first. > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> *Upgrading to 3.5.0* > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> Upgrading a running ZooKeeper > > > > ensemble > > > > >>> to > > > > >>> > >>> 3.5.0 > > > > >>> > >>> >> >> should be > > > > >>> > >>> >> >> > >>>> done > > > > >>> > >>> >> >> > >>>>>> only > > > > >>> > >>> >> >> > >>>>>>>>> after > > > > >>> > >>> >> >> > >>>>>>>>>>> upgrading your ensemble to > > the > > > > 3.4.6 > > > > >>> > release. > > > > >>> > >>> >> Note > > > > >>> > >>> >> >> that > > > > >>> > >>> >> >> > >>> this > > > > >>> > >>> >> >> > >>>> is > > > > >>> > >>> >> >> > >>>>>>> only > > > > >>> > >>> >> >> > >>>>>>>>>>> necessary for rolling > > upgrades > > > > (if > > > > >>> you're > > > > >>> > >>> fine > > > > >>> > >>> >> with > > > > >>> > >>> >> >> > >>> shutting > > > > >>> > >>> >> >> > >>>>> down > > > > >>> > >>> >> >> > >>>>>>> the > > > > >>> > >>> >> >> > >>>>>>>>>>> system completely, you don't > > > > have to go > > > > >>> > >>> through > > > > >>> > >>> >> >> 3.4.6). > > > > >>> > >>> >> >> > >> If > > > > >>> > >>> >> >> > >>>> you > > > > >>> > >>> >> >> > >>>>>>>> attempt > > > > >>> > >>> >> >> > >>>>>>>>> a > > > > >>> > >>> >> >> > >>>>>>>>>>> rolling upgrade without going > > > > through > > > > >>> 3.4.6 > > > > >>> > >>> (for > > > > >>> > >>> >> >> example > > > > >>> > >>> >> >> > >>> from > > > > >>> > >>> >> >> > >>>>>>> 3.4.5), > > > > >>> > >>> >> >> > >>>>>>>>> you > > > > >>> > >>> >> >> > >>>>>>>>>>> may get the following error: > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 > > > [myid:2] > > > > - INFO > > > > >>> > >>> >> [localhost/ > > > > >>> > >>> >> >> > >>>>>> 127.0.0.1:2784 > > > > >>> > >>> >> >> > >>>>>>>>>>> > > :QuorumCnxManager$Listener@498 > > > ] > > > > - > > > > >>> Received > > > > >>> > >>> >> >> connection > > > > >>> > >>> >> >> > >>>> request > > > > >>> > >>> >> >> > >>>>> / > > > > >>> > >>> >> >> > >>>>>>>>>>> 127.0.0.1:60876 > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> 2013-01-30 11:32:10,663 > > > [myid:2] > > > > - WARN > > > > >>> > >>> >> [localhost/ > > > > >>> > >>> >> >> > >>>>>> 127.0.0.1:2784 > > > > >>> > >>> >> >> > >>>>>>>>>>> :QuorumCnxManager@349] - > > > Invalid > > > > >>> server > > > > >>> > id: > > > > >>> > >>> >> -65536 > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> During a rolling upgrade, > > each > > > > server > > > > >>> is > > > > >>> > >>> taken > > > > >>> > >>> >> down > > > > >>> > >>> >> >> in > > > > >>> > >>> >> >> > >> turn > > > > >>> > >>> >> >> > >>>> and > > > > >>> > >>> >> >> > >>>>>>>>> rebooted > > > > >>> > >>> >> >> > >>>>>>>>>>> with the new 3.5.0 binaries. > > > > Before > > > > >>> > starting > > > > >>> > >>> the > > > > >>> > >>> >> >> server > > > > >>> > >>> >> >> > >>> with > > > > >>> > >>> >> >> > >>>>>> 3.5.0 > > > > >>> > >>> >> >> > >>>>>>>>>>> binaries, we highly recommend > > > > updating > > > > >>> the > > > > >>> > >>> >> >> configuration > > > > >>> > >>> >> >> > >>> file > > > > >>> > >>> >> >> > >>>>> so > > > > >>> > >>> >> >> > >>>>>>> that > > > > >>> > >>> >> >> > >>>>>>>>> all > > > > >>> > >>> >> >> > >>>>>>>>>>> server statements > > > "server.x=..." > > > > >>> contain > > > > >>> > >>> client > > > > >>> > >>> >> >> ports > > > > >>> > >>> >> >> > >> (see > > > > >>> > >>> >> >> > >>>> the > > > > >>> > >>> >> >> > >>>>>>>> section > > > > >>> > >>> >> >> > >>>>>>>>>>> Specifying > > > > >>> > >>> >> >> > >>>>>>>>>>> the client port). As > > explained > > > > earlier > > > > >>> you > > > > >>> > >>> may > > > > >>> > >>> >> >> leave the > > > > >>> > >>> >> >> > >>>>>>>> configuration > > > > >>> > >>> >> >> > >>>>>>>>>> in a > > > > >>> > >>> >> >> > >>>>>>>>>>> single file, as well as leave > > > the > > > > >>> > >>> >> >> > >>>> clientPort/clientPortAddress > > > > >>> > >>> >> >> > >>>>>>>>> statements > > > > >>> > >>> >> >> > >>>>>>>>>>> (although if you specify > > client > > > > ports > > > > >>> in > > > > >>> > the > > > > >>> > >>> new > > > > >>> > >>> >> >> format, > > > > >>> > >>> >> >> > >>>> these > > > > >>> > >>> >> >> > >>>>>>>>> statements > > > > >>> > >>> >> >> > >>>>>>>>>>> are now redundant). > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> Could you please let me know > > > > about this > > > > >>> > case. > > > > >>> > >>> >> >> Appreciate > > > > >>> > >>> >> >> > >> if > > > > >>> > >>> >> >> > >>>>>> respond > > > > >>> > >>> >> >> > >>>>>>>>> soon. > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>>> Thanks, > > > > >>> > >>> >> >> > >>>>>>>>>>> --------------------- > > > > >>> > >>> >> >> > >>>>>>>>>>> Kuldeep Singh Budania > > > > >>> > >>> >> >> > >>>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>>> > > > > >>> > >>> >> >> > >>>>>>>> > > > > >>> > >>> >> >> > >>>>>>> > > > > >>> > >>> >> >> > >>>>>> > > > > >>> > >>> >> >> > >>>>> > > > > >>> > >>> >> >> > >>>> > > > > >>> > >>> >> >> > >>> > > > > >>> > >>> >> >> > >> > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> ________________________________ > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> CONFIDENTIALITY & PRIVACY NOTICE > > > > >>> > >>> >> >> This e-mail (including any attachments) is > > > strictly > > > > >>> > >>> confidential > > > > >>> > >>> >> and > > > > >>> > >>> >> >> may also contain privileged information. If you are > > > > not the > > > > >>> > >>> intended > > > > >>> > >>> >> >> recipient you are not authorised to read, print, > > > save, > > > > >>> process or > > > > >>> > >>> >> disclose > > > > >>> > >>> >> >> this message. If you have received this message by > > > > mistake, > > > > >>> > please > > > > >>> > >>> >> inform > > > > >>> > >>> >> >> the sender immediately and destroy this e-mail, its > > > > >>> attachments > > > > >>> > >>> and any > > > > >>> > >>> >> >> copies. Any use, distribution, reproduction or > > > > disclosure by > > > > >>> any > > > > >>> > >>> person > > > > >>> > >>> >> >> other than the intended recipient is strictly > > > > prohibited and > > > > >>> the > > > > >>> > >>> person > > > > >>> > >>> >> >> responsible may incur in penalties. > > > > >>> > >>> >> >> The use of this e-mail is only for professional > > > > purposes; > > > > >>> > >>> there is > > > > >>> > >>> >> no > > > > >>> > >>> >> >> guarantee that the correspondence towards this > > e-mail > > > > will be > > > > >>> > read > > > > >>> > >>> >> only by > > > > >>> > >>> >> >> the recipient, because, under certain > > circumstances, > > > > there > > > > >>> may > > > > >>> > be a > > > > >>> > >>> >> need to > > > > >>> > >>> >> >> access this email by third subjects belonging to > > the > > > > Company. > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> ________________________________ > > > > >>> > >>> >> >> > > > > >>> > >>> >> >> CONFIDENTIALITY & PRIVACY NOTICE > > > > >>> > >>> >> >> This e-mail (including any attachments) is strictly > > > > >>> confidential > > > > >>> > >>> and > > > > >>> > >>> >> may > > > > >>> > >>> >> >> also contain privileged information. If you are not > > > the > > > > >>> intended > > > > >>> > >>> >> recipient > > > > >>> > >>> >> >> you are not authorised to read, print, save, > > process > > > or > > > > >>> disclose > > > > >>> > >>> this > > > > >>> > >>> >> >> message. If you have received this message by > > > mistake, > > > > please > > > > >>> > >>> inform > > > > >>> > >>> >> the > > > > >>> > >>> >> >> sender immediately and destroy this e-mail, its > > > > attachments > > > > >>> and > > > > >>> > any > > > > >>> > >>> >> copies. > > > > >>> > >>> >> >> Any use, distribution, reproduction or disclosure > > by > > > > any > > > > >>> person > > > > >>> > >>> other > > > > >>> > >>> >> than > > > > >>> > >>> >> >> the intended recipient is strictly prohibited and > > the > > > > person > > > > >>> > >>> >> responsible > > > > >>> > >>> >> >> may incur in penalties. > > > > >>> > >>> >> >> The use of this e-mail is only for professional > > > > purposes; > > > > >>> there > > > > >>> > is > > > > >>> > >>> no > > > > >>> > >>> >> >> guarantee that the correspondence towards this > > e-mail > > > > will be > > > > >>> > read > > > > >>> > >>> >> only by > > > > >>> > >>> >> >> the recipient, because, under certain > > circumstances, > > > > there > > > > >>> may > > > > >>> > be a > > > > >>> > >>> >> need to > > > > >>> > >>> >> >> access this email by third subjects belonging to > > the > > > > Company. > > > > >>> > >>> >> >> > > > > >>> > >>> >> > > > > > >>> > >>> >> > > > > >>> > >>> > > > > > >>> > >>> > > > > >>> > >> > > > > >>> > > > > > >>> > > > > >> > > > > > > > > > > > > ________________________________ > > > > > > > > CONFIDENTIALITY & PRIVACY NOTICE > > > > This e-mail (including any attachments) is strictly confidential and > > may > > > > also contain privileged information. If you are not the intended > > > recipient > > > > you are not authorised to read, print, save, process or disclose this > > > > message. If you have received this message by mistake, please inform > > the > > > > sender immediately and destroy this e-mail, its attachments and any > > > copies. > > > > Any use, distribution, reproduction or disclosure by any person other > > > than > > > > the intended recipient is strictly prohibited and the person > > responsible > > > > may incur in penalties. > > > > The use of this e-mail is only for professional purposes; there is no > > > > guarantee that the correspondence towards this e-mail will be read only > > > by > > > > the recipient, because, under certain circumstances, there may be a > > need > > > to > > > > access this email by third subjects belonging to the Company. > > > > > > > > >
