So, Does that mean only one RDD is created by all receivers?
On Sun, Dec 20, 2015 at 10:23 PM, Saisai Shao <sai.sai.s...@gmail.com> wrote: > Normally there will be one RDD in each batch. > > You could refer to the implementation of DStream#getOrCompute. > > > On Mon, Dec 21, 2015 at 11:04 AM, Arun Patel <arunp.bigd...@gmail.com> > wrote: > >> It may be simple question...But, I am struggling to understand this.... >> >> DStream is a sequence of RDDs created in a batch window. So, how do I >> know how many RDDs are created in a batch? >> >> I am clear about the number of partitions created which is >> >> Number of Partitions = (Batch Interval / spark.streaming.blockInterval) >> * number of receivers >> >> Is it like one RDD per receiver? or Multiple RDDs per receiver? What is >> the easiest way to find it? >> >> Arun >> > >
