Repository: apex-malhar Updated Branches: refs/heads/master 0f8442472 -> 1bd113921
APEXMALHAR-2305 updated documentation on SessionWindow to reflect the new proto-session window semantics Project: http://git-wip-us.apache.org/repos/asf/apex-malhar/repo Commit: http://git-wip-us.apache.org/repos/asf/apex-malhar/commit/1bd11392 Tree: http://git-wip-us.apache.org/repos/asf/apex-malhar/tree/1bd11392 Diff: http://git-wip-us.apache.org/repos/asf/apex-malhar/diff/1bd11392 Branch: refs/heads/master Commit: 1bd113921a0f25b7e09a84514f7020bce3dd9aba Parents: 0f84424 Author: David Yan <[email protected]> Authored: Wed Oct 19 18:13:21 2016 -0700 Committer: David Yan <[email protected]> Committed: Thu Oct 20 11:06:29 2016 -0700 ---------------------------------------------------------------------- .../windowedOperator/session-windows-1.png | Bin 16868 -> 15139 bytes .../windowedOperator/session-windows-2.png | Bin 15823 -> 14238 bytes .../windowedOperator/session-windows-3.png | Bin 13323 -> 11613 bytes .../windowedOperator/session-windows-4.png | Bin 0 -> 15176 bytes docs/operators/windowedOperator.md | 20 +++++++++++-------- 5 files changed, 12 insertions(+), 8 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-1.png ---------------------------------------------------------------------- diff --git a/docs/operators/images/windowedOperator/session-windows-1.png b/docs/operators/images/windowedOperator/session-windows-1.png index 505ad6f..78712e8 100644 Binary files a/docs/operators/images/windowedOperator/session-windows-1.png and b/docs/operators/images/windowedOperator/session-windows-1.png differ http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-2.png ---------------------------------------------------------------------- diff --git a/docs/operators/images/windowedOperator/session-windows-2.png b/docs/operators/images/windowedOperator/session-windows-2.png index 2fee78e..19bfab0 100644 Binary files a/docs/operators/images/windowedOperator/session-windows-2.png and b/docs/operators/images/windowedOperator/session-windows-2.png differ http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-3.png ---------------------------------------------------------------------- diff --git a/docs/operators/images/windowedOperator/session-windows-3.png b/docs/operators/images/windowedOperator/session-windows-3.png index 14f0ccd..e7810eb 100644 Binary files a/docs/operators/images/windowedOperator/session-windows-3.png and b/docs/operators/images/windowedOperator/session-windows-3.png differ http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/images/windowedOperator/session-windows-4.png ---------------------------------------------------------------------- diff --git a/docs/operators/images/windowedOperator/session-windows-4.png b/docs/operators/images/windowedOperator/session-windows-4.png new file mode 100644 index 0000000..43838c8 Binary files /dev/null and b/docs/operators/images/windowedOperator/session-windows-4.png differ http://git-wip-us.apache.org/repos/asf/apex-malhar/blob/1bd11392/docs/operators/windowedOperator.md ---------------------------------------------------------------------- diff --git a/docs/operators/windowedOperator.md b/docs/operators/windowedOperator.md index 0549b89..021842e 100644 --- a/docs/operators/windowedOperator.md +++ b/docs/operators/windowedOperator.md @@ -81,36 +81,40 @@ windowedOperator.setWindowOption(new WindowOption.TimeWindows(Duration.standardM ### `SessionWindows` -`SessionWindow`s have variable durations and are based on the key of the tuple. Each tuple is assigned to exactly one window. It takes a duration parameter `minGap`, which specifies the minimum time gap between two `SessionWindow`s of the same key. +`SessionWindow`s have variable durations and are based on the key of the tuple. Each tuple is assigned to exactly one window. It takes a duration parameter `minGap`, which specifies the minimum time gap between two tuples that belong to two *different* `SessionWindows` of the same key. `minGap` is also the duration of the "proto-session" window for a single timestamp, and it is the minimum duration of any session window. ```java // Setting a session window option with a minimum gap of one hour windowedOperator.setWindowOption(new WindowOption.SessionWindows(Duration.standardHours(1))); ``` -To ensure that no two `SessionWindow`s of the same key are less than `minGap` apart, upon an arrival of a tuple, the `WindowedOperator` does the following checks: +Upon arrival of a tuple, a proto-session window is created. A proto-session window for a tuple is a temporary session window with begin timestamp being the tuple timestamp and the duration being `minGap`. -#### The new tuple can be assigned to an existing `SessionWindow` without change + + +To ensure that there are no two tuples of the same key in different session windows that are less than `minGap` apart, the `WindowedOperator` does the following checks: + +#### The proto-session window can be fit into an existing `SessionWindow` of the same key without change The new tuple is simply applied to the state of the existing `SessionWindow`.  -#### The arrival of the new tuple would result in a merge of the two existing session windows +#### The proto-session window overlaps with two existing session windows of the same key -A new Session Window is created with the merged state of the two existing `SessionWindow`s of the same key, plus the new tuple. The two existing `SessionWindow`s will be deleted and retraction triggers for the two deleted windows will be fired. (Please see [here](#triggers) for details on `Trigger`s) +A new Session Window is created with the merged state of the two existing `SessionWindow`s, plus the new tuple. The two existing `SessionWindow`s will be deleted and retraction triggers for the two deleted windows will be fired. (Please see [here](#triggers) for details on `Trigger`s)  -#### The arrival of the new tuple would result in an extension of an existing `SessionWindow`s +#### The proto-session window overlaps with one existing session window of the same key -A new `SessionWindow` is created with the state of the existing `SessionWindow`, plus the new tuple, with a longer duration than the existing `SessionWindow` to cover the new tuple. The existing `SessionWindow` will be deleted and a retraction trigger for the old window will be fired. +A new `SessionWindow` is created with the state of the existing `SessionWindow`, plus the new tuple, with a longer duration than the existing `SessionWindow` and possibly an earlier begin timestamp to cover the new tuple. The existing `SessionWindow` will be deleted and a retraction trigger for the old window will be fired.  #### All of the above checks return false -A new `SessionWindow` is created for the new tuple. +The proto-session window is in effect and the new tuple is assigned to that window. ## Timestamp Extractor
