[ https://issues.apache.org/jira/browse/SPARK-12206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075508#comment-15075508 ]
Shixiong Zhu commented on SPARK-12206: -------------------------------------- Because only batches that are multiple times of slide duration will run, "StreamingJobProgressListener" cannot get info of other batches. /cc [~tdas] what do you think about this one? > Streaming WebUI shows incorrect batch statistics when using Window operations > ----------------------------------------------------------------------------- > > Key: SPARK-12206 > URL: https://issues.apache.org/jira/browse/SPARK-12206 > Project: Spark > Issue Type: Bug > Components: Streaming > Affects Versions: 1.5.1 > Reporter: Anand Iyer > Priority: Minor > Attachments: streaming-webui.png > > > I have a streaming app that uses the Window(...) function to create a sliding > window, and perform transformations on the window'd DStream. > The Batch statistics section of the Streaming UI starts displaying stats for > each Window, instead of each micro-batch. Is that expected behavior? > The "Input Size" column shows incorrect values. The streaming application is > receiving about 1K events/sec. However, the "Input Size" column shows values > in the single digits or low double digits. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org