Re: Stream reading from database using spark streaming

ok that is fine. so the source is an IMDB something like Oracle TimesTen that I have worked with before. The second source is some organised data (I assume you mean structured tabular data 1. Data is read from source one, the IMDB. The assumption is that within the batch interval that data

Re: Stream reading from database using spark streaming

I don't understand this. How are you going to read from RDBMS database, through JDBC? How often are you going to sample the transactional tables? You may find that a JDBC connection will take longer than your sliding window length. Is this for real time analytics? Thanks Dr Mich Talebzadeh

Re: Stream reading from database using spark streaming

http://www.sparkexpert.com/2015/03/28/loading-database-data-into-spark-using-data-sources-api/ https://spark.apache.org/docs/1.6.1/api/scala/index.html#org.apache.spark.rdd.JdbcRDD FYI On Thu, Jun 2, 2016 at 6:26 AM, Zakaria Hili wrote: > I want to use spark streaming to

Stream reading from database using spark streaming

I want to use spark streaming to read data from RDBMS database like mysql. but I don't know how to do this using JavaStreamingContext JavaStreamingContext jssc = new JavaStreamingContext(conf, Durations.milliseconds(500));DataFrame df = jssc. ?? I search in the internet but I didn't find