Yes, basically from the currently implementation it should be. On Mon, Dec 21, 2015 at 6:39 PM, Arun Patel <[email protected]> wrote:
> So, Does that mean only one RDD is created by all receivers? > > > > On Sun, Dec 20, 2015 at 10:23 PM, Saisai Shao <[email protected]> > wrote: > >> Normally there will be one RDD in each batch. >> >> You could refer to the implementation of DStream#getOrCompute. >> >> >> On Mon, Dec 21, 2015 at 11:04 AM, Arun Patel <[email protected]> >> wrote: >> >>> It may be simple question...But, I am struggling to understand this.... >>> >>> DStream is a sequence of RDDs created in a batch window. So, how do I >>> know how many RDDs are created in a batch? >>> >>> I am clear about the number of partitions created which is >>> >>> Number of Partitions = (Batch Interval / spark.streaming.blockInterval) >>> * number of receivers >>> >>> Is it like one RDD per receiver? or Multiple RDDs per receiver? What is >>> the easiest way to find it? >>> >>> Arun >>> >> >> >
