Oh, didn't know about optimize so I'll definitely keep that in mind. The reason I was asking about primary shards is that I saw, when starting from a rsync'd datafolder off of one of the nodes, double the amount of documents. It wasn't immediatly apparent but when I later on tried with two rsyncs matching up old node 1 with new node 1 and old node 2 with new node 2 the "duplicates" went away... and the cluster recovered significantly faster. But reading this, it seems to be sufficient just to rsync the data folder from any 1 node in the old cluster and things will just work? Is there a way to verify the consistency of my cluster? Something like index checksums, or somesuch?
On 24 October 2014 17:54, Ivan Brusic <[email protected]> wrote: > Unless you are moving to new hardware, there is no need to rsync your > data. Both Elasticsaerch 0.90.x and 1.3.x are based on Lucene 4, so the > underlying data is compatible. Of course, you should backup your data > before such an upgrade. > > After restarting your new cluster with your old data, I would run an > optimize on your indices so that Lucene can upgrade all your segments into > the new format. There have been some issues with Lucene format > incompatibilities, but they usually deal with indices with beta Lucene > versions. > > You cannot bring up a mixed cluster between 0.90 and 1.x, so you would > need to stop all your VMs. Why are you interested in primary shards? > Elasticsearch is not like most database where the primary node has an extra > special connotation. I have not played around with shard allocation much, > but here is an old article: > http://blog.sematext.com/2012/05/29/elasticsearch-shard-placement-control/ > > Cheers, > > Ivan > > On Thu, Oct 23, 2014 at 4:18 PM, Magnus Persson < > [email protected]> wrote: > >> Ah, slight typo in regard to the old cluster. It is 1 replica per index. >> >> >> On Thursday, October 23, 2014 10:13:57 PM UTC+2, Magnus Persson wrote: >>> >>> So I'm about to upgrade to 1.3.4, but due to some unfortunate >>> circumstances I need to migrate my ES cluster to new VMs. >>> The environment is fairly simple. At the top I have logstash agent >>> pulling messages off of a Redis server and feeding it to my 2 node cluster >>> (2 replicas, 2 shards per index). So for what it's worth I can stop >>> logstash and the cluster will essentially stop indexing data, allowing me >>> to shut it down without issue. Once I have the old cluster shut down, I >>> intend to rsync it over to the new cluster which is 3 nodes (2 replicas, 3 >>> shards per index). >>> What is the best approach here? I was thinking that I could rsync the >>> data folder from 1 of my 2 VMs running on the old cluster but then I >>> realized that the primary shard for each index might not be on that VM. Can >>> I manually set the primary shard somehow? >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/ee5aa6d1-3339-4d45-8cd6-76614269e501%40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/ee5aa6d1-3339-4d45-8cd6-76614269e501%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/elasticsearch/8MWsKqDIKpA/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQB_R8bj9mNSASWJVpGZwR5JYJSdu6bk_5DvzxPgtbU-Bg%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQB_R8bj9mNSASWJVpGZwR5JYJSdu6bk_5DvzxPgtbU-Bg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACjthGbFU8CMUZQSZmQkLbaLgtMpXMQfyUDjO9LEBnfcb29ThQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
