>From ZK documentation, observers do not participate in vote, so Pushkar, when you said 5 nodes participate in voting, what exactly you mean?
-- Observers are non-voting members of an ensemble which only hear the results of votes, not the agreement protocol that leads up to them. Per ZK documentation, 3.4 includes observers, does that mean Jan thought experiment is practically possible, correct? On Fri, May 26, 2017 at 3:53 AM, Rick Leir <rl...@leirtech.com> wrote: > Jan, Shawn, Susheel > > First steps first. First, let's do a fault-tolerant cluster, then maybe a > _geographically_ fault-tolerant cluster. > > Add another server in either DC1 or DC2, in a separate rack, with > independent power etc. As Shawn says below, install the third ZK there. You > would satisfy most of your requirements that way. > > cheers -- Rick > > > On 2017-05-23 12:56 PM, Shawn Heisey wrote: > >> On 5/23/2017 10:12 AM, Susheel Kumar wrote: >> >>> Hi Jan, FYI - Since last year, I have been running a Solr 6.0 cluster in >>> one of lower env with 6 shards/replica in dc1 & 6 shard/replica in dc2 >>> (each shard replicated cross data center) with 3 ZK in dc1 and 2 ZK in dc2. >>> (I didn't have the availability of 3rd data center for ZK so went with only >>> 2 data center with above configuration) and so far no issues. Its been >>> running fine, indexing, replicating data, serving queries etc. So in my >>> test, setting up single cluster across two zones/data center works without >>> any issue when there is no or very minimal latency (in my case around 30ms >>> one way >>> >> >> With that setup, if dc2 goes down, you're all good, but if dc1 goes down, >> you're not. >> >> There aren't enough ZK servers in dc2 to maintain quorum when dc1 is >> unreachable, and SolrCloud is going to go read-only. Queries would most >> likely work, but you would not be able to change the indexes at all. >> >> ZooKeeper with N total servers requires int((N/2)+1) servers to be >> operational to maintain quorum. This means that with five total servers, >> three must be operational and able to talk to each other, or ZK cannot >> guarantee that there is no split-brain, so quorum is lost. >> >> ZK in two data centers will never be fully fault-tolerant. There is no >> combination of servers that will work properly. You must have three data >> centers for a geographically fault-tolerant cluster. Solr would be >> optional in the third data center. ZK must be installed in all three. >> >> Thanks, >> Shawn >> >> >
