Hi Arthur, I have done something similar years ago when I was working for a newspaper. We kept articles in database and full text was done with external program. There was a trigger the tables with articles that on every change operation adds record in a queue table. Something like this: article_id, opetation_type, table_name Then there was a cron jon every minute that reads from this table and - On delete deletes the entry - On Update deletes the entry and generates new simple page with the new artice – only title and content and put it on indexer to be indexed - On insert generates new simple page with the new artice – only title and content and put it on indexer to be indexed
Articles have been placed in some directory like this: /<root_dir>/<table_name>/<id>/<content>.html. Then this path was returned and easy parsed to generate appropriate link to the artice. After success removes respective record from the queue and we have near realtime This can be done with ES but much easier best reragards, Nickolay Kolev 08 януари 2014, сряда, 21:13:35 UTC+2, arthurX написа: > > Hello! I use MySQL as my primary datastore and use Elasticsearch to > further index the documents. > My problem is keeping the data in ES in sync with MySQL. > > Currently I have two methods in mind: > 1. whenever add or update an entry in MySQL, do the action together in ES. > 2. Do some cron jobs that periodically keep ES in sync with the data in > MySQL. > > For method 2 I wonder how can I check if an entry is already indexed in > Elasticsearch. And would it be efficient at all if I have to check every > entry to see if it is updated? > > I am new to the technology and I am afraid I had missed some really > obvious and established solutions here. Or otherwise the "normal" way this > situation is handled? > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7dff496c-cc26-4620-bf1b-115a53f0ca6d%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
