Well, the scheduling delay is the time a batch has to wait for getting resources. So even if there is no backlog in processing and scheduling delay is 0, there is one batch that is being processed at any point of time, which explains the difference.
On Tue, Jun 30, 2015 at 2:42 AM, bit1...@163.com <bit1...@163.com> wrote: > Hi, Spark users, > > Following images are copied from spark streaming UI. I observed for about > 30 minutes, and see that the Processed records(438768, at the moment I > copied the image) are always lagging behind Received records(480783) by > about 40k records, > Since the waiting batches is 1 and the received records are many more > than processed records, I can't understand why the total delay or > scheduling day is not obvious（5 secs) here. > > Can someone help explain what clues from this UI? > Thanks. > > > > > > ------------------------------ > bit1...@163.com >
