Correct, this would create new indices.
On Apr 21, 2014, at 2:50 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
> If I understand this correct with this pattern evey restore is a new index?
>
> curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore" -d '{
> "indices": "index_1,index_2",
> "ignore_unavailable": "true",
> "include_global_state": false,
> "rename_pattern": "index_(.+)",
> "rename_replacement": "restored_index_$1"
> }'
>
> On Mon, Apr 21, 2014 at 2:32 PM, Andrew Selden
> <andrew.sel...@elasticsearch.com> wrote:
> Hi,
>
> How many indices you create is really up to you. Typically you would make a
> snapshot of an index once and then schedule incremental snapshots at regular
> intervals.
>
> Andrew
>
> On Apr 21, 2014, at 2:24 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
>
>> Thanks for the information. To keep the indexes in near real time sync I'll
>> end up creating thousands of indexes every day alone with this method,
>> correct?
>>
>> On Mon, Apr 21, 2014 at 1:37 PM, Andrew Selden
>> <andrew.sel...@elasticsearch.com> wrote:
>> Hi,
>>
>> I can see how the documentation may not be as clear as it should be on this
>> point. Let me try to explain. If you refer back to [1], you will see an
>> example restore command:
>> $ curl -XPOST "localhost:9200/_snapshot/my_backup/snapshot_1/_restore" -d '{
>>
>>
>> "indices": "index_1,index_2",
>>
>>
>> "ignore_unavailable": "true",
>>
>>
>> "include_global_state": false,
>>
>>
>> "rename_pattern": "index_(.+)",
>>
>>
>> "rename_replacement": "restored_index_$1"
>>
>>
>> }'
>> The important parts to notice are: rename_pattern and rename_replacement.
>> What this means is that, for any index in the repository named with the
>> prefix 'index_', it will be restored to the cluster under the new name
>> 'restored_index_$1', where $1 is the name of the original index. So
>> "index_1" gets restored to "restored_index_1", meanwhile you can keep
>> index_1 open and servicing requests.
>>
>> The basic operational workflow is that you would perform the restore
>> operation with a rename. Once the restore is complete, do some sanity
>> checking to confirm that you have in fact restored what you intended to.
>> Finally, to make "restored_index_1" live and take "index_1" offline without
>> impacting production service, you would use an atomic rename alias command
>> [2].
>>
>> This assumes you have been using aliases to manage your indices. For
>> example, typically you would set up an alias to point to a 'logical' index
>> to mask one or more 'physical' indices. This is highly recommended as it
>> gives you the flexibility to do things like hot-swap restored/live indices
>> as in the above example.
>>
>> Hope that helps.
>>
>> Andrew
>>
>> 1.
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html
>> 2.
>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-aliases.html
>>
>> On Apr 21, 2014, at 12:36 PM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
>>
>>> This document seem to suggest that to do the restore elasticsearch need to
>>> close the index, which means index is not available during that time while
>>> restore is occurring?
>>>
>>> On Mon, Apr 21, 2014 at 12:31 PM, Andrew Selden
>>> <andrew.sel...@elasticsearch.com> wrote:
>>> Hi,
>>>
>>> Snapshot/restore is fine to run on an active, production system. The
>>> indices are still available for normal indexing and searching operations.
>>> You can throttle the snapshot operation if you are concerned about
>>> impacting production quality of service [1].
>>>
>>> I would suggest avoiding using scroll as a backup strategy. Keeping a
>>> scroll open for a long time can be taxing on an actively used production
>>> cluster. You are much better off using the snapshot/restore functionality.
>>>
>>> Andrew
>>>
>>> 1.
>>> http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-snapshots.html
>>>
>>>
>>> On Apr 21, 2014, at 9:23 AM, Mohit Anchlia <mohitanch...@gmail.com> wrote:
>>>
>>>> Our data centers are a actively used and need to be highly available. Does
>>>> snapshot and restore happen in the background and indexes are still
>>>> available to the clients for read/write?
>>>>
>>>> Can Scroll be used to read indexes continuously and insert/update them?
>>>>
>>>> On Mon, Apr 21, 2014 at 5:57 AM, Alexander Reelsen <a...@spinscale.de>
>>>> wrote:
>>>> hey,
>>>>
>>>> snapshot and restore might be an option, if you are willing to accept a
>>>> lag a few minutes behind. another option might be to index/update/delete
>>>> all your data into two different clusters, so you dont need replication at
>>>> all?
>>>>
>>>>
>>>> --Alex
>>>>
>>>>
>>>> On Fri, Apr 18, 2014 at 10:19 PM, Mohit Anchlia <mohitanch...@gmail.com>
>>>> wrote:
>>>> As I understand there is currently no feature that does async replication
>>>> between 2 clusters or even within the same cluster, but we have a need to
>>>> write one. What would be the best way to do it in elasticsearch? I was
>>>> thinking of leveraging Scroll for this.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWrUdDdiqy62yHaaS6bJJ08_txDCNNXR8rGr%3DRGY8gAv-Q%40mail.gmail.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/CAGCwEM9G8539fUmrzLWgkXoSYAEyUYkjUbyjeBiNVhBdarWTFA%40mail.gmail.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "elasticsearch" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpqVdoSWv552vh3ZOL7siBiMUCGRUV5NcBwAFoWPisVLQ%40mail.gmail.com.
>>>>
>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/B852C0E8-A2BE-4C23-B14A-EE2593A4D7ED%40elasticsearch.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "elasticsearch" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to elasticsearch+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWr14cNv05GAjoxtXVtUY4-uPzaxt200tp_spDNXvO_tyg%40mail.gmail.com.
>>>
>>> For more options, visit https://groups.google.com/d/optout.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elasticsearch" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to elasticsearch+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elasticsearch/C9E4589A-FBCB-4FC3-9492-35B8A722EB20%40elasticsearch.com.
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/9AC0A79F-B695-4F7D-80FC-D39C9FD2F15E%40elasticsearch.com.
>
> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to elasticsearch+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/CAOT3TWpGTUH3cqrsqeGTvPucQpRTY%3D0%2BHYB_YRQ8tFqJQ1myCQ%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to elasticsearch+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8EEA285B-9B46-4DDC-A9C9-B82134AA0D8A%40elasticsearch.com.
For more options, visit https://groups.google.com/d/optout.
