Ola, Thanks Mark! >> Does this work even when outside clients (apps for indexing or searching)
> send their requests directly to individual nodes? >> Let's use the example from my email where we end up with 2 groups of > nodes: 7-node group with 2 ZK nodes on the same network and 3-node group with > 1 > ZK node on the same network. > > The 3-node group with 1 ZK would not have a functioning zk - so it would stop > accepting updates. If it could serve a complete view of the index, it would > though, for searches. So in this case information in this 1 ZK node would tell the 3 Solr nodes whether they have all index data or if some shards are missing (i.e. were only on nodes in the other 7-node group)? And if nodes figure out they don't have all index data they will reject search requests? Or will they accept and perform searches, but return responses that tell the client that the searched index was not complete? > The 7-node group would have a working ZK it could talk to, and it would > continue > to accept updates as long as a node for a shard for that hash range is up. It > would also of course serve searches. Right, so if the node for the shard where a doc is supposed to go to is in that 3-node group, then the indexing request will be rejected. Is this correct? > In this case, hitting a box in the 3-node group for searches would start > becoming stale. A smart client would no longer hit those boxes though. > > If you have a 'dumb' client or load balancer, then yes - you would have > to remove the bad nodes from rotation. Aha, yes and yes. > We could improve this or make the behavior configurable. At least initially > though, we figured it was better if we kept serving searches even when we > cannot > talk to zookeeper. Makes sense. Do responses carry something to alert the client that "something is rotten in the state of cluster"? Thanks, Otis ---- Performance Monitoring for Solr / ElasticSearch / HBase - http://sematext.com/spm