Short of using a river to feed both indexes the same stream of updates updates I doubt that you will find any solution. Good news is why it's tedious as you pointed out, once setup it flows very smoothly. We use RabbitMQ in our case.
A possible future feature would be an API within index creation that allows shadowing the indexing of one or more other indexes, without having to go through duplication client side or using a river. When the shadowed index goes away so could the shadowing, or api call could delete the shadow. cheers, Rob On Friday, February 21, 2014 12:11:52 AM UTC-8, Andrew Kane wrote: > > I tried to post a reply yesterday but it looks like it never made it. > > Thank you all for the quick replies. Here's a slightly better explanation > of where I believe the race condition occurs. > > When the scan/scroll starts, the alias is still pointing to the old index, > so updates go to the old index. Let's say you update Document 1. If the > scroll/scan has already passed Document 1, the new index never sees the > update. The three solutions you mentioned Nik are to either: > > 1. Keep track of updates manually [tedious] > 2. Pause the jobs that perform the updates [out of sync] > 3. Send updates to both indexes [also tedious] > > However, none of these seem ideal. > > - Andrew > > On Tuesday, February 18, 2014 8:41:18 PM UTC-8, Andrew Kane wrote: >> >> Hi, >> >> I've followed the documentation for zero-downtime mapping changes and it >> works great. >> http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/ >> >> However, there is a (pretty big) race condition with this approach - >> while reindexing, changes may not make it to the new index. I've looked >> all over and haven't found a single solution to address this. The best >> attempt I've seen is to buffer updates, but this is tedious and still >> leaves a race condition (with a smaller window). My initial thoughts were >> to create a write alias that points to the old and new indices and use >> versioning. However, there is no way to write to multiple indices >> atomically. >> >> It seems like this issue should affect most Elasticsearch users (whether >> they realize it or not). Does anyone have a good solution to this? >> >> Thanks, >> Andrew >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/01041af1-3032-41d0-9b80-c6861aa3d6dc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
