[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=339662&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-339662
]
ASF GitHub Bot logged work on BEAM-8575:
----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Nov/19 01:06
Start Date: 07/Nov/19 01:06
Worklog Time Spent: 10m
Work Description: liumomo315 commented on pull request #9957: [BEAM-8575]
Add validates runner tests for 1. Custom window fn: Test a customized window fn
work as expected; 2. Windows idempotency: Applying the same window fn (or
window fn + GBK) to the input multiple times will have the same effect as
applying it once.
URL: https://github.com/apache/beam/pull/9957#discussion_r343407158
##########
File path: sdks/python/apache_beam/transforms/window_test.py
##########
@@ -252,6 +276,50 @@ def test_timestamped_with_combiners(self):
assert_that(mean_per_window, equal_to([(0, 2.0), (1, 7.0)]),
label='assert:mean')
+ @attr('ValidatesRunner')
+ def test_custom_windows(self):
+ with TestPipeline() as p:
+ result = (p | Create([TimestampedValue(x, x * 100) for x in range(7)])
+ | 'custom window' >> WindowInto(TestCustomWindows(250, 200))
+ | 'insert key' >> Map(lambda v: ('key', v.value))
+ | GroupByKey())
+
+ assert_that(result, equal_to([('key', [0, 1, 2]),
+ ('key', [3, 4]),
+ ('key', [5]),
+ ('key', [6])]))
+
+ @attr('ValidatesRunner')
+ def test_windows_idempotency(self):
+ with TestPipeline() as p:
+ result = (p | Create([(x, x * 2) for x in range(5)])
Review comment:
I tried that but it failed.
Dug this issue more and found this is a known issue with Python timestamped
value, see https://github.com/apache/beam/pull/8206. In short, to have the
downstream ptransforms treat an timestamped element as an element with a
timestamp instead of a timestamped value object, there must be a DoFn (could be
a simple Map(lamda x: x)) applied to the timestamped elements. (This should be
fixed, but probably not in this PR....)
I found this test class already have a helper fun timestamped_key_values, so
I reuse it in the new tests.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 339662)
Remaining Estimate: 0h
Time Spent: 10m
> Add more Python validates runner tests
> --------------------------------------
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
> Reporter: wendy liu
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
