Wow!

Am amazed how performant a restore is compared to restarting a populated 
cluster!

Existing cluster:
5 nodes, all master and data eligible
Configured, 5shards,0replica/index
Apache log data
330 indices
4361031 documents
   
current configuration to prevent cluster from considering re-allocation 
until all nodes is the following (set to same number as total nodes in 
cluster)

gateway.recover_after_nodes: 5
  
When restarting this cluster, I usually have trouble getting all nodes to 
stay online during restart. This typically causes shard re-allocation, 
followed possibly by more re-allocation.
Typically, minimally it takes 10 min for the cluster to recognize all 5 
nodes
Followed by minimally 40 minutes (often more) to go green.

When restoring this cluster from snapshot
Since there is no data in the new cluster, 5 nodes are recognized almost 
immmediately (no 10 min wait)
Restore (not completely successful, see below) is only about 20 minutes.
Not even a hint that any node was unhappy and might drop from the cluster.

Noticed - Shard initialization is 20 shards at a time, twice as many as 
when restarting a populated cluster.

But, looks like two shards remain unallocated
- A replica of a Marvel index (the Primary appears to have successfully 
initialized)
- A primary of one of the Apache log indices.

Last time I experienced this, I simply deleted the problem index and 
re-input the data (BTW - Input is by month, so to restore the one day 
involves input 30 days. For the other 29 days, the version was <not> 
incremented which better answers a post to this Forum weeks ago when I 
asked about this scenario)

In this scenario, I don't think I can re-load historical Marvel data (does 
it even exist in a way that can be re-loaded?)
Also, am wondering if there is a better solution to "fixing" a single index 
that refuses to initialize.

TIA,
Tony






   

On Friday, February 21, 2014 4:56:13 PM UTC-8, Tony Su wrote:
>
> Cool.
>
> Still not knowing whether the act of registering a snapshot repository 
> pointing to a populated directory might "initialize/wipe" the location, I 
> made a copy before registering again.
>
> After registering the repository, I queried for existing snapshots and all 
> looks well, so it looks like registering is non-destructive.
>
> Tony
>
>
>
>
> On Friday, February 21, 2014 3:48:51 PM UTC-8, Binh Ly wrote:
>>
>> When you originally defined the snapshot, you did something like this (or 
>> similar):
>>
>> PUT http://localhost:9200/_snapshot/Snapshots
>> {
>>     "type": "fs",
>>     "settings": {
>>         "location": "/blah"
>>     }
>> }
>>
>> On the new/empty cluster, this snapshot is not yet registered, so the 
>> first thing you need to do is to run that again:
>>
>> PUT http://localhost:9200/_snapshot/Snapshots
>> {
>>     "type": "fs",
>>     "settings": {
>>         "location": "/blah"
>>     }
>> }
>>
>> Then after that, you should be able to restore to the new/empty cluster 
>> (as long as that location "/blah" is accessible from the new cluster node).
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to [email protected].
To view this discussion on the web visit 
https://groups.google.com/d/msgid/elasticsearch/b1cd7695-bfce-4d57-bdc7-bac09b039071%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Reply via email to