Spark Streaming - use the data in different jobs

2015-10-18 Thread Oded Maimon
duled jobs access same rdd's? - should we keep the data to hbase and read it from other jobs? - other ways? I believe that the answer will be using external db/storage.. hoping to have a different solution :) Thanks. Regards, Oded Maimon Scene53. -- *This email and any files tran

Re: Few basic spark questions

2015-07-14 Thread Oded Maimon
o if i use it, how can i read the data from a different spark scala/java application? how do i find/query this data? Regards, Oded Maimon Scene53. On Tue, Jul 14, 2015 at 12:35 AM, Feynman Liang wrote: > Sorry; I think I may have used poor wording. SparkR will let you use R to > analyze

Re: Few basic spark questions

2015-07-12 Thread Oded Maimon
any help / idea will be appreciated :) thanks Regards, Oded Maimon Scene53. On Sun, Jul 12, 2015 at 4:49 PM, Oded Maimon wrote: > Hi All, > we are evaluating spark for real-time analytic. what we are trying to do > is the following: > >- READER APP- use custom receiver t

Few basic spark questions

2015-07-12 Thread Oded Maimon
Hi All, we are evaluating spark for real-time analytic. what we are trying to do is the following: - READER APP- use custom receiver to get data from rabbitmq (written in scala) - ANALYZER APP - use spark R application to read the data (windowed), analyze it every minute and save the r