/foreachRDD equivalent for pyspark Structured Streaming
Local Time: 3 May 2017 12:05 PM
UTC Time: 3 May 2017 10:05
From: tathagata.das1...@gmail.com
To: peay
user@spark.apache.org
You can apply apply any kind of aggregation on windows. There are some built in
aggregations (e.g. sum and count) as well as
You can apply apply any kind of aggregation on windows. There are some
built in aggregations (e.g. sum and count) as well as there is an API for
user-defined aggregations (scala/Java) that works with both batch and
streaming DFs.
See the programming guide if you havent seen it already
- windowing -
Hello,
I would like to get started on Spark Streaming with a simple window.
I've got some existing Spark code that takes a dataframe, and outputs a
dataframe. This includes various joins and operations that are not supported by
structured streaming yet. I am looking to essentially map/apply thi
