With a shard with "listening" status and some logic on the mechanism that does the load balancing between replicas, we can achieve the goal.
The SPLITSHARD action makes replicas from the original shard which are in "inactive" state, this shards buffering the updates and when the operation ends, the parent shard becomes "inactive" and the new replicas are promoted to "active" state. Like "inactive" state, we can have a "listening" state that never becomes "active", unless a leader election operation happen and this shard with "listening" status be the unique that is alive. In addition, is necessary add new metadata to the shard on clusterstate.json file to mark that replica as a replica with replication purposes, and resigns when other replica becomes active. -- Yago Riveiro Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Thursday, October 24, 2013 at 2:16 PM, Toke Eskildsen wrote: > On Thu, 2013-10-24 at 13:27 +0200, yriveiro wrote: > > The motivation of this is simple, I want have replication but I don't want > > have n replicas actives with full resources allocated (cache and so on). > > This is usefull in enviroments where replication is needed but a high query > > throughput is not fundamental and the resources are limited. > > > > > Coincidentally we recently talked about the exact same setup. > > We are looking at sharding a 20 TB index into 20 * 1 TB shards, each > located on their own dedicated physical SSD, which has more than enough > horsepower for our needs. For replication, we have a remote storage > system capable of serving requests for 2-4 shards with acceptable > latency. > > Projected performance for the SSD setup is superior (5-10 times) to our > remote storage, so we would like to hit only the SSDs if possible. > Setting up a cloud to issue all requests to the SSD-shards unless a > catastrophic failure happened to on of them and in that case fallback to > the remote story replica for only that shard, would be perfect. > > > I know that right now is not possible, but I think that it's a feature that > > can be implemented in a easy way creating a new status for shards. > > > > > shardIsLastResort=true? On paper it seems like a simple addition, but I > am not at familiar enough with the SolrCloud-code to guess if it is easy > to implement. > > - Toke Eskildsen, State and University Library, Denmark
