Hello Sanjay,
Yes, your understanding of lazy semantics is correct. But ideally
every batch should read based on the batch interval provided in the
StreamingContext. Can you open a JIRA on this?
On Mon, Mar 24, 2014 at 7:45 AM, Sanjay Awatramani
wrote:
> Hi All,
>
> I found out why this problem
Hi All,
I found out why this problem exists. Consider the following scenario:
- a DStream is created from any source. (I've checked with file and socket)
- No actions are applied to this DStream
- Sliding Window operation is applied to this DStream and an action is applied
to the sliding window.
Hi,
I want to run a map/reduce process over last 5 seconds of data, every 4
seconds. This is quite similar to the sliding window pictorial example under
Window Operations section on
http://spark.incubator.apache.org/docs/latest/streaming-programming-guide.html
.
The RDDs returned by window t
