Re: Zookeeper leader election takes a long time.

2016-10-13 Thread Anand Parthasarathy
Hi Michael, We have reproduced this issue on a private AWS setup that has public IP access. I will send you the details of the instance IP and the credentials separately. If it needs to be shared with more people, I am happy to share with them as well. Thanks Anand. On Tue, Oct 11, 2016 at 3:46

Re: Zookeeper leader election takes a long time.

2016-10-13 Thread Anand Parthasarathy
Just wanted to let you know that at this time, one of the node is powered off and the other two nodes took more than 10 minutes to converge. Our script exits and so, we don't know when it exactly converged. Normally, it takes < 100 seconds to converge. Thanks, Anand. On Thu, Oct 13, 2016 at 10:09

Re: outstandingChanges queue grows without bound

2016-10-13 Thread Arshad Mohammad
Hi Mike I also faced same issue. There is test patch in ZOOKEEPER-2570 which can be used to quickly check performance gains in each modification. Hope it is useful. -Arshad On Thu, Oct 13, 2016 at 1:27 AM, Mike Solomon wrote: > I've been performance testing 3.5.2 and hit an interesting unavai

Re: outstandingChanges queue grows without bound

2016-10-13 Thread Edward Ribeiro
Very interesting patch, Mike. I've left a couple of review comments (hope you don't mind) in the https://github.com/msolo/zookeeper/commit/75da352d506c2e3b0001d28acc058c 422b3c8f0c commit. :) Cheers, Eddie On Thu, Oct 13, 2016 at 4:06 PM, Arshad Mohammad < arshad.mohamma...@gmail.com> wrote: >

Re: Zookeeper leader election takes a long time.

2016-10-13 Thread Ryan Zhang
Hi, Anand, I took a look and I wonder how do you explain the following The N1.log starts at around 2016-10-03 12:37:38,073 [myid:1] - INFO [QuorumPeer[myid=1]/127.0.0.1:5002:QuorumPeer@714] - LOOKING and it failed to talk to Node 2 (id: 2) 2016-10-03 12:37:38,136 [myid:1] - WARN [WorkerSender

Re: Zookeeper leader election takes a long time.

2016-10-13 Thread Ryan Zhang
Btw, also, from n3.log it looks like id: 3 actually started to follow id:2 016-10-03 12:39:18,682 [myid:3] - INFO [WorkerReceiver[myid=3]:FastLeaderElection@597] - Notification: 1 (message format version), 2 (n.leader), 0x20002bfb5 (n.zxid), 0x7 (n.round), LEADING (n.state), 2 (n.sid), 0x3 (n.

Re: Zookeeper leader election takes a long time.

2016-10-13 Thread Michael Han
>> It looks similar to ZOOKEEPER-2164 but there it is a connection timeout where Node 2 is not reachable. It sounds to me that issue described in this email is the same as ZOOKEEPER-2164 after checking the attached log of node 1 and node 3. Let's check a specific time window for both logs, for exa

Re: Zookeeper leader election takes a long time.

2016-10-13 Thread Michael Han
>> it started a new round but then I seem to see the election messages from Node 2 again. Any idea why? My guess is node 2 is back online and ZK service was started. In this use case node 2 does not get offline permanently, IIUC. On Thu, Oct 13, 2016 at 3:41 PM, Ryan Zhang wrote: > Hi, Anand,

Extending session lifetime with client requests

2016-10-13 Thread Peng Li
Hi there, Suppose a zookeeper session timeout is configured to 10 seconds. If at time t, a client sends a getData() request to the ensemble and succeeds, can I assume the client's zookeeper session will be live for the next (10 - delta) seconds according to the client's local clock? In other words