Hi In spark streaming based on receivers - when receiver gets data and store in blocks for workers to process, How many blocks does receiver gives to worker.
Say I have a streaming app with 30 sec of batch interval what will happen 1.for first batch(first 30 sec) there will not be any data for worker to process, only receiver will fetch data and store in blocks 2.for second batch- worker will work on block fetched in step1 and receibvers will fetch more data Is this understanding correct ? In that case I have worst case of 2*batch interval delay in event processing . 2.Also what if worker is slow - say in above example of 30 sec batch interval - worker took 2 min to process a batch then in next batch will it get 4 blocks(2 min data fetch by receiver) or just the 1 batch interval data irrespective of its speed. Thanks