Then you can look at leveraging Logstash with a broker (redis/rabbitmq), it'll do NRT and provides that queue.
There may be other community based plugins that can do this, otherwise it's something you'd have to handle yourself. On 15 January 2015 at 12:29, Todd Nine <[email protected]> wrote: > Hey Mark, > We're looking for a solution that's faster than a snapshot+restore. We > do follow an eventual consistency model, but we'd like the replication to > be near realtime, so a snapshot and restore would be too slow. We really > need something that synchronizes between regions with some sort of log or > queue, so that if a region goes down, we can replay all missed operations. > We were going to put an SQS queue in our application tier, but if this is > something an ES plugin can provide, we would prefer to utilize that over > writing our own. > > I was searching the plugin api, but I can't seem to find anything that > will receive post document write/delete events I can hook into. Ideally, I > would like each node to queue up changes to primary shards. This way, we > can distribute sending the changes among all our nodes and we won't have a > single point of failure. > > Thanks, > Todd > > On Wed, Jan 14, 2015 at 3:03 PM, Mark Walkom <[email protected]> wrote: > >> You could use snapshot and restore, or even Logstash. >> >> On 15 January 2015 at 10:07, Todd Nine <[email protected]> wrote: >> >>> Hi all, >>> We have a deployment scenario I can't seem to find any examples of, >>> and any help would be greatly appreciated. We're running ElasticSearch in >>> 3 AWS regions. We want these regions to survive a failure from other >>> regions, and we want all writes and reads from our clients to occur in the >>> local region. Rather than have 1 large cluster that spans 3 regions, I >>> would like the ability to asynchronously replicate document creation and >>> deletion across the regions. Our application creates immutable documents, >>> so update semantics don't apply. Is there any way this can be accomplished >>> currently? Note that I don't think a river will provide us with the >>> partition we need since it is a singleton. >>> >>> Thanks! >>> Todd >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/874b63bf-4cf5-44c9-923d-33134cd4d234%40googlegroups.com >>> <https://groups.google.com/d/msgid/elasticsearch/874b63bf-4cf5-44c9-923d-33134cd4d234%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "elasticsearch" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/elasticsearch/nYv8slw-0j4/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_fs2ykJsqvL9GVOn9dN-snh_dN0KkggtGH9mSOj0ha9w%40mail.gmail.com >> <https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_fs2ykJsqvL9GVOn9dN-snh_dN0KkggtGH9mSOj0ha9w%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CA%2Byzqf9HzLB%3Ds9NnhCv0bRBXGdjFc%3DJqd_0MJJdR1vmHOH3USA%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CA%2Byzqf9HzLB%3Ds9NnhCv0bRBXGdjFc%3DJqd_0MJJdR1vmHOH3USA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEYi1X9uPFs5_F1SH_Vt1Efe%2BeZczD15vHimXXTvh_nSHD36Kg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
