+1 having a queue and consumers between your source of truth and ES is a great approach. You cab decouple and independently scale ( and stop when needed as DP said) the different components, minimising impact to your users. On 09/01/2014 7:35 AM, "David Pilato" <[email protected]> wrote:
> I would do 1/ to have a more near real time search. > Also, I'd the idea that I have an object in memory and I simply push it to > MySQL and to ES in the same time. No need to read again the object from > MySQL to index it in another process (proposition 2) > > That said you could use also a Message Queue in the middle if you want to > be able at some point to stop your ES cluster without stopping your > application. > This is what I did in the past. > > My 2 cents > > -- > *David Pilato* | *Technical Advocate* | *Elasticsearch.com* > @dadoonet <https://twitter.com/dadoonet> | > @elasticsearchfr<https://twitter.com/elasticsearchfr> > > > Le 8 janvier 2014 at 20:13:40, arthurX > ([email protected]<//[email protected]>) > a écrit: > > Hello! I use MySQL as my primary datastore and use Elasticsearch to > further index the documents. > My problem is keeping the data in ES in sync with MySQL. > > Currently I have two methods in mind: > 1. whenever add or update an entry in MySQL, do the action together in ES. > 2. Do some cron jobs that periodically keep ES in sync with the data in > MySQL. > > For method 2 I wonder how can I check if an entry is already indexed in > Elasticsearch. And would it be efficient at all if I have to check every > entry to see if it is updated? > > I am new to the technology and I am afraid I had missed some really > obvious and established solutions here. Or otherwise the "normal" way this > situation is handled? > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/55d842e5-277f-4d24-b5a9-8be5b5544dbc%40googlegroups.com > . > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/etPan.52cdb688.70a64e2a.1449b%40MacBook-Air-de-David.local > . > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CACj2-4LRefoG1u0MMtX96UhoJG72mXHk9U9G2w4Gv4XJtB9aLg%40mail.gmail.com. For more options, visit https://groups.google.com/groups/opt_out.
