RE: Spark streaming updating a large window more frequently

2015-05-08 Thread Mohammed Guller
updating a large window more frequently Hi, I am pretty new to spark/spark_streaming so please excuse my naivety. I have streaming event stream that is timestamped and I would like to aggregate it into, let's say, hourly buckets. Now the simple answer is to use a window operation with window leng

Spark streaming updating a large window more frequently

2015-05-08 Thread Ankur Chauhan
Hi, I am pretty new to spark/spark_streaming so please excuse my naivety. I have streaming event stream that is timestamped and I would like to aggregate it into, let's say, hourly buckets. Now the simple answer is to use a window operation with window length of 1 hr and sliding interval of 1hr