[
https://issues.apache.org/jira/browse/BEAM-8045?focusedWorklogId=301111&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-301111
]
ASF GitHub Bot logged work on BEAM-8045:
----------------------------------------
Author: ASF GitHub Bot
Created on: 26/Aug/19 09:14
Start Date: 26/Aug/19 09:14
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #9406: [BEAM-8045]
Custom windows patterns
URL: https://github.com/apache/beam/pull/9406#discussion_r317519636
##########
File path: website/src/documentation/patterns/custom-window.md
##########
@@ -0,0 +1,112 @@
+---
+layout: section
+title: "Custom window patterns"
+section_menu: section-menu/documentation.html
+permalink: /documentation/patterns/custom-window/
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+# Custom window patterns
+The samples on this page demonstrate common custom window patterns. You can
create custom windows with [`WindowFn` functions]({{ site.baseurl
}}/documentation/programming-guide/#windowing#provided-windowing-functions).
For more information, see the [programming guide section on windowing]({{
site.baseurl }}/documentation/programming-guide/#windowing).
+
+**Note**: Merging windows isn't supported in Python.
+
+## Using data to dynamically set session window gaps
+
+You can modify the
[`assignWindows`](https://beam.apache.org/releases/javadoc/current/index.html?org/apache/beam/sdk/transforms/windowing/SlidingWindows.html)
function to use data-driven gaps instead of fixed windows.
+
+Access the `assignWindows` function through
`WindowFn.AssignContext.element()`. The original, fixed-duration
`assignWindows` function is:
+
+```java
+{% github_sample
/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/snippets/Snippets.java
tag:CustomSessionWindow1
+%}
+```
+
+### Creating data-driven gaps
+To use data-driven gaps, add the following snippets to the `assignWindows`
function:
+- A default value for cases where the custom gap is not present in the data
+- A way to set the attribute from the main pipeline as a method of the custom
windows.
+
+For example, the following function assigns each element to a window between
the timestamp and the gapDuration:
+
+```java
+{% github_sample
/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/snippets/Snippets.java
tag:CustomSessionWindow3
+%}
+```
+
+In this function, the `withDefaultGapDuration` and `withGapAttribute` methods
are:
+
+```java
+{% github_sample
/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/snippets/Snippets.java
tag:CustomSessionWindow4
+%}
+```
+
+Then, the new `gapAttribute` field and constructor dynamically create session
windows with the calculated gap duration:
+
+```java
+{% github_sample
/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/snippets/Snippets.java
tag:CustomSessionWindow2
+%}
+```
+
+### Windowing messages into sessions
+After creating data-driven gaps, you can window Pub/Sub messages into the new,
custom sessions:
+
+```java
+{% github_sample
/apache/beam/blob/master/examples/java/src/main/java/org/apache/beam/examples/snippets/Snippets.java
tag:CustomSessionWindow6
+%}
+```
+
+### Example data and windows
+The following test data has some messages with and without the `gap` attribute.
+
+```
+.apply("Create data", Create.timestamped(
+
TimestampedValue.of("{\"user\":\"user-1\",\"score\":\"12\",\"gap\":\"5\"}", new
Instant()),
+ TimestampedValue.of("{\"user\":\"user-2\",\"score\":\"4\"}", new
Instant()),
+
TimestampedValue.of("{\"user\":\"user-1\",\"score\":\"-3\",\"gap\":\"5\"}", new
Instant().plus(2000)),
+
TimestampedValue.of("{\"user\":\"user-1\",\"score\":\"2\",\"gap\":\"5\"}", new
Instant().plus(9000)),
+
TimestampedValue.of("{\"user\":\"user-1\",\"score\":\"7\",\"gap\":\"5\"}", new
Instant().plus(12000)),
+ TimestampedValue.of("{\"user\":\"user-2\",\"score\":\"10\"}", new
Instant().plus(12000)))
+ .withCoder(StringUtf8Coder.of()))
+```
+
+The diagram below visualizes the test data:
+
+![Two sets of data and the standard and dynamic sessions with which the data
is windowed.]( {{ "/images/standard-vs-dynamic-sessions.png" | prepend:
site.baseurl }})
+
+#### Standard sessions
+
+Standard sessions use the following windows and scores:
+```
+user=user-2, score=4,
window=[2019-05-26T13:28:49.122Z..2019-05-26T13:28:59.122Z)
+user=user-1, score=18,
window=[2019-05-26T13:28:48.582Z..2019-05-26T13:29:12.774Z)
+user=user-2, score=10,
window=[2019-05-26T13:29:03.367Z..2019-05-26T13:29:13.367Z)
+```
+
+User #1 sees two events separated by 12 seconds. With standard sessions, the
gap defaults to 10 seconds; both scores are in different sessions, so the
scores aren't added.
+
+User #2 sees four events, seperated by two, seven, and three seconds,
respectively. Since none of the gaps are greater than the default, the four
events are in the same standard session and added together (18 points).
+
+#### Dyanmic sessions
Review comment:
Dynamic
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 301111)
Time Spent: 20m (was: 10m)
> Publish custom windows pattern
> ------------------------------
>
> Key: BEAM-8045
> URL: https://issues.apache.org/jira/browse/BEAM-8045
> Project: Beam
> Issue Type: Improvement
> Components: website
> Reporter: Cyrus Maden
> Assignee: Cyrus Maden
> Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
