Re: reduceByKey Not Being Called by Spark Streaming

2014-07-03 Thread Dan H. 
Hi All,

I was able to resolve this matter with a simple fix.  It seems that in order
to process a reduceByKey and the flat map operations at the same time, the
only way to resolve was to increase the number of threads to  1.

Since I'm developing on my personal machine for speed, I simply updated the
sparkURL argument to:
   private static String sparkURL = local[2];  //Instead of local

,which is then used by the JavaStreamingContext method as a parameter.

After I made this change, I was able to see the reduceByKey values properly
aggregated and counted.

Best Regards,

D



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/reduceByKey-Not-Being-Called-by-Spark-Streaming-tp8684p8739.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

reduceByKey Not Being Called by Spark Streaming

2014-07-02 Thread Dan H. 
: Stream 0 received 0 blocks
14/07/02 16:54:10 INFO scheduler.JobScheduler: Added jobs for time
140434165 ms




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/reduceByKey-Not-Being-Called-by-Spark-Streaming-tp8684.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.