"I've been exploring the option of setting every shard to have N replicas where N is the number of nodes in the cluster."
You would *always* have an allocated shard. Do you mean N-1 replicas? That's not much better of an idea, since the loss of a single node would then leave you with an unallocated shard. On Wednesday, July 9, 2014 5:57:53 PM UTC-4, [email protected] wrote: > > Hi all, > > I'm considering using elasticsearch as a repository for a PoC I'm > currently developing. > > This PoC models an application that needs durability but not isolability, > so I'm fine with the eventual consistency of reads against the most recent > writes. > > As durability is paramount (we can't affort to lose the data unless 100% > of the nodes die) I've been exploring the option of setting every shard to > have N replicas where N is the number of nodes in the cluster. > > From what I've read so far it is possible to dynamically set the number or > replicas which triggers a replication throttled replication process. > > I would like to have some help on the following steps (I'm running ES in > embedded mode in a Java application): > > 1 - How can I set the number or replicas using the native Java client ? > 2 - What happens if a node dies and the number of replicas is lowered to > the number of surviving ones? > 3 - Is it possible, from a participating node, to access the list of nodes > in the cluster so I can use their count to set the number of replicas (step > 1) ? > 4 - is it possible to hook a callback to the event of a node joining or > leaving the cluster ? > > I envisioning the following mechanism: > > a) - Start with one node, a given number of shards and 1 replica > b)- Each time a node joins I adjust the number or replicas to match the > new node count. In this case, there would be 2 replicas > c) - An arbitrary number of nodes might be added and I'd execute step b) > accordingly > d) - At any time a node might leave the cluster and thus I need to lower > the number or replicas to the new node count (I assume that the cluster > would go ahead and proceed to compensate the lost replica by asking an > existing node to hold 2 replicas instead of one; is this stopped by > lowering the number or replicas?) > > > The ultimate goal is to make sure no data is loss unless 100% of the nodes > die before a new one can acquire a full replica. > > Is this doable? Does this make sense at all ? > > For the time being, I'm not worried about lack of disk space or bandwidth > as I'm still in the very early days of the PoC. > > Thank you very much for all your work and help. > > Gonçalo > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/b0c7e6ba-2845-40ea-9c50-6723566538e8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
