The definitions of "real time" are quite vague. I also tend to use "real time" for sub-second latencies with a guarantee of a maximum response time. Other just mean by "real time" everything that is not batching, pushing data over the wire as the changes happen, no matter how large the change is - a process which can take huge delays in worst case, even minutes.
Although it is possible to run a select query every second with JDBC plugin crontab, this puts a heavy load on the system with not much gain, so I don't recommend it. For instant replication, one would have to extend JDBC plugin with a special queue on the side of the RDBMS and use proprietary driver API. Even with such a queue the latency is often higher than a few seconds. Jörg On Wed, Jan 7, 2015 at 5:52 PM, David Pilato <[email protected]> wrote: > Well. I did not mention your plugin because he basically asked for "I want > to know how can I input the data that I'm getting in Elasticsearch in real > time.". > Real time aspect here seems to be important. > > Elasticsearch is not real time but near real time. > JDBC river is obviously not real time as IIRC it runs a job every x > minutes or so. > > > > -- > *David Pilato* | *Technical Advocate* | *Elasticsearch.com > <http://Elasticsearch.com>* > @dadoonet <https://twitter.com/dadoonet> | @elasticsearchfr > <https://twitter.com/elasticsearchfr> | @scrutmydocs > <https://twitter.com/scrutmydocs> > > > > Le 7 janv. 2015 à 17:35, [email protected] a écrit : > > You can use the JDBC plugin instead of reinventing the wheel. > > https://github.com/jprante/elasticsearch-river-jdbc > > Jörg > > On Wed, Jan 7, 2015 at 5:28 PM, Marian Valero <[email protected]> > wrote: > >> Thanks! >>> >> >> I have a problem with jsonBuilder() method, eclipse don't found this, I >> do it that way: >> >> public static void main(String args[]) throws SQLException { >> >> int count=0; >> Node node = nodeBuilder().client(true).node(); >> Client client = node.client(); >> >> String url = "jdbc:oracle:thin:@eso:1521:eso"; >> >> Properties props = new Properties(); >> props.setProperty("user", "eso"); >> props.setProperty("password", "eso"); >> >> Connection conn = DriverManager.getConnection(url,props); >> >> String sql ="select * from responselog"; >> >> PreparedStatement preStatement = conn.prepareStatement(sql); >> >> ResultSet result = preStatement.executeQuery(); >> >> // Timestamp timestamp = null; >> >> BulkRequestBuilder bulkRequest = client.prepareBulk(); >> >> while(result.next() && count<1000){ >> count++; >> String id = result.getString("id"); >> String deliveryid = result.getString("deliveryid"); >> String msgid = result.getString("msgid"); >> String rspdate = result.getString("rspdate"); >> String parsedresponse = result.getString("parsedresponse"); >> String shortcode = result.getString("shortcode"); >> String insid = result.getString("insid"); >> String mobilenumber = result.getString("mobilenumber"); >> String rawresponse = result.getString("rawresponse"); >> >> XContentBuilder builder = jsonBuilder() >> .startObject() >> .field("deliveryid", deliveryid) >> .field("msgid", msgid) >> .field("message", "trying out Elasticsearch") >> .endObject(); >> >> >> // YOUR ORACLE stuff here >> // either use client#prepare, or use Requests# to directly build >> index/delete requests >> bulkRequest.add(client.prepareIndex("logs", "responselog", id) >> .setSource(jsonBuilder() >> .startObject() >> .field("deliveryid", deliveryid) >> .field("msgid", msgid) >> .field("rspdate", rspdate) >> .field("parsedresponse", parsedresponse) >> .field("shortcode", shortcode) >> .field("insid", insid) >> .field("mobilenumber", mobilenumber) >> .field("rawresponse", rawresponse) >> .endObject() >> ) >> ); >> >> System.out.println("Responselog from Oracle : "+ >> result.getString("id")+" "+result.getString("deliveryid")+" >> "+result.getString("msgid")+" "+result.getString("rspdate")+" >> "+result.getString("parsedresponse")+" "+result.getString("shortcode")+" >> "+result.getString("insid")+" "+result.getString("mobilenumber")+" >> "+result.getString("rawresponse")); >> } >> >> BulkResponse bulkResponse = bulkRequest.execute().actionGet(); >> System.out.println("done"); >> } >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com >> <https://groups.google.com/d/msgid/elasticsearch/865f9ddc-517e-4e1b-aba4-8d7e2a2eebf1%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com > <https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFcx7gXvXzg83cu9dFx7%3DYtW%2BA_K%2BtBANUpdD-86uB8GQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr > <https://groups.google.com/d/msgid/elasticsearch/AA7A49A7-5BFC-4DFD-9745-BBEAAD60E603%40pilato.fr?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoE0CUreVbvpQrtXE186yqjA%2BtiYPCfZkwK_3QxfoOnOSA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
