Mario Briggs created SPARK-13650: ------------------------------------ Summary: Usage of the window() function on DStream Key: SPARK-13650 URL: https://issues.apache.org/jira/browse/SPARK-13650 Project: Spark Issue Type: Bug Components: Streaming Affects Versions: 1.6.0, 1.5.2, 2.0.0 Reporter: Mario Briggs Priority: Minor
Is there some guidance of the usage of the Window() function on DStream. Here is my academic use-case for which it fails. Standard word count val ssc = new StreamingContext(sparkConf, Seconds(6)) val messages = KafkaUtils.createDirectStream(...) val words = messages.map(_._2).flatMap(_.split(" ")) val window = words.window(Seconds(12), Seconds(6)) window.count().print() For the first batch interval it gives the count and then it hangs (inside the unionRDD) I say the above use-case is academic since one can achieve similar fuctionality by using instead the more compact API words.countByWindow(Seconds(12), Seconds(6)) which works fine. Is the first approach above not the intended way of using the .window() API -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org