This is a little late but. I agree with the third option, if you can. But you should publish to a queue and then that goes to ES. That way you can use a 2 phase commit to the DB and to the queue.
Even with option 1 and 2 ... use a queue. Talend has built in stuff to do this because it is more than just ETL (it is an ESB where ETL is just a use case) On Saturday, January 11, 2014 1:47:40 AM UTC-5, David Pilato wrote: > > IMHO, some options: > > * ETL tool like Talend > * a batch process you need to write > * modify application which create those data and push to Elasticsearch > from it > > I prefer the latter as your search engine will be well synchronized with > your database. > Also, it will be a gain, database performance wise, as you don't need to > read again your DB to inject into ES. > > My 2 cents > > -- > David ;-) > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > > Le 11 janv. 2014 à 04:12, jsp <[email protected] <javascript:>> a > écrit : > > Thanks Karl, > Do you have any recommendation for alternatives to using rivers. i want > to read data from sql db/tables. > thanks > jay > > On Friday, 10 January 2014 13:55:29 UTC-8, Karol Gwaj wrote: >> >> Consider the fact that rivers could be deprecated in future versions of >> elasticsearch: >> >> http://www.linkedin.com/groups/Official-guide-writing-ElasticSearch-rivers-3393294.S.268274223 >> >> >> >> On Wednesday, January 8, 2014 4:26:58 PM UTC, jsp wrote: >>> >>> Hi, >>> I am looking at implementing ES to index & query data that I get from my >>> SQL Server databases/tables. I was initially using river to read data from >>> Sql server tables but one of the developers in my team recommended looking >>> at using logstash. Can anyone comment to any benefits of using one over >>> another? I have not been able to find any documentation regarding reading >>> data from SQL server using logstash <http://logstash.net/docs/1.3.2/>. >>> Can someone point me to a guide on how to get started with logstash & sql >>> server. >>> Thanks >>> J >>> >> -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected] <javascript:>. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/890a3acc-74f9-4999-b42d-790c78a97c39%40googlegroups.com > . > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9cf23220-b281-4b38-92c8-bcbfb8dd3df5%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
