Re: Wrong leader election leads to shard removal

2013-08-16 Thread Ido Kissos
Yes, I have erased the tlog in replica 2 and it appears that the the first replica's tlog was corrupted because of an ungracefull servlet shutdown. There was no log for it unfortunately, neither the zookeeper log logged anything about this. Is there a a place I could check in the zookeeper what

Re: Wrong leader election leads to shard removal

2013-08-16 Thread Erick Erickson
bq:why does it replicate all the index instead of copying just the newer formed segments because there's no guarantee that the segments are identical on the nodes that make up a shard. The simplest way to conceptualize this is to consider the autocommit settings on the servers Let's say the hard

Re: Wrong leader election leads to shard removal

2013-08-14 Thread Manuel Le Normand
Does this sound like the scenario that happened: By removing the index dir from replica 2 I also removed the tlog from which the zookeeper extracts the version of the two replicas and decides which one should be elected to leader. As replica 2 did have no tlog, the zk didn't have anyway to compare

Re: Wrong leader election leads to shard removal

2013-08-14 Thread Mark Miller
On Aug 14, 2013, at 9:01 AM, Manuel Le Normand manuel.lenorm...@gmail.com wrote: Does this sound like the scenario that happened: By removing the index dir from replica 2 I also removed the flog Did you also remove the tlog dir? It's normally: data/index data/tlog from which the