[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=340757&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-340757
]
ASF GitHub Bot logged work on BEAM-8575:
----------------------------------------
Author: ASF GitHub Bot
Created on: 08/Nov/19 21:40
Start Date: 08/Nov/19 21:40
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #9957: [BEAM-8575]
Add validates runner tests for 1. Custom window fn: Test a customized window fn
work as expected; 2. Windows idempotency: Applying the same window fn (or
window fn + GBK) to the input multiple times will have the same effect as
applying it once.
URL: https://github.com/apache/beam/pull/9957#discussion_r344377337
##########
File path: sdks/python/apache_beam/transforms/window_test.py
##########
@@ -65,6 +76,44 @@ def process(self, element, window=core.DoFn.WindowParam):
reify_windows = core.ParDo(ReifyWindowsFn())
+class TestCustomWindows(NonMergingWindowFn):
+ """A custom non merging window fn which assigns elements into interval
windows
+ based on the element timestamps.
+ """
+
+ def __init__(self, first_window_end, second_window_end):
+ self.first_window_end = Timestamp.of(first_window_end)
+ self.second_window_end = Timestamp.of(second_window_end)
+
+ def assign(self, context):
+ timestamp = context.timestamp
+ if timestamp < self.first_window_end:
+ return [IntervalWindow(0, self.first_window_end)]
+ elif timestamp < self.second_window_end:
+ return [IntervalWindow(self.first_window_end, self.second_window_end)]
+ else:
+ return [IntervalWindow(self.second_window_end, timestamp)]
+
+ def get_window_coder(self):
+ return IntervalWindowCoder()
+
+ def to_runner_api_parameter(self, context):
Review comment:
It doesn't seem like there is a good way for custom user window fns to
return a python dict or some other structure and you currently need to
translate the data you want to store in the window fn into some proto. Instead
of defining a test windowfn proto, use one of the well known protobuf types
such as
[Struct](https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/struct)
or
[BytesValue](https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/bytes-value)
and embed the information within those objects yourself instead of defining
your own within StandardWindowFns.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 340757)
Time Spent: 1h 40m (was: 1.5h)
> Add more Python validates runner tests
> --------------------------------------
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
> Reporter: wendy liu
> Assignee: wendy liu
> Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
