Github user tdas commented on a diff in the pull request:
https://github.com/apache/spark/pull/4875#discussion_r25981446
--- Diff:
streaming/src/main/scala/org/apache/spark/streaming/dstream/DStream.scala ---
@@ -649,7 +649,23 @@ abstract class DStream[T: ClassTag] (
* DStream's batching interval
*/
def window(windowDuration: Duration, slideDuration: Duration):
DStream[T] = {
- new WindowedDStream(this, windowDuration, slideDuration)
+ new WindowedDStream(this, windowDuration, slideDuration, None)
+ }
+
+ /**
+ * Return a new DStream in which each RDD contains all the elements in
seen in a
+ * sliding window of time over this DStream.
+ * @param windowDuration width of the window; must be a multiple of this
DStream's
+ * batching interval
+ * @param slideDuration sliding interval of the window (i.e., the
interval after which
+ * the new DStream will generate RDDs); must be a
multiple of this
+ * DStream's batching interval
+ * @param initialWindow initial window values to prepend starting with
the oldest entry
--- End diff --
It would be good to specify in the docs, that initial window is an ordered
list of RDDs, and each RDD represents data from a batch.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
