[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=345714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-345714
]
ASF GitHub Bot logged work on BEAM-8575:
----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Nov/19 00:29
Start Date: 19/Nov/19 00:29
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10070:
[BEAM-8575] Added a unit test for Reshuffle to test that Reshuffle pr…
URL: https://github.com/apache/beam/pull/10070#discussion_r347678737
##########
File path: sdks/python/apache_beam/transforms/util_test.py
##########
@@ -276,6 +279,13 @@ def process(self, element):
with self.assertRaisesRegex(ValueError, r'window.*None.*add_timestamps2'):
pipeline.run()
+class AddTimestamp(beam.DoFn):
+ def process(self, element, timestamp=beam.DoFn.TimestampParam):
+ yield beam.window.TimestampedValue(element, timestamp)
Review comment:
My understanding is that it should be a no-op, if Reshuffle preserves
timestamps. This is what this test is testing.
Its Java parity is the testReshufflePreservesTimestamps in file
beam/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ReshuffleTest.java
which wrapped the string element into TimestampedValue twice.
The first time the element becomes TimestampedValue<String>;
the second time the element becomes
TimestampedValue<TimestampedValue<String>>.
Python doesn't have nested TimestampedValue type and doesn't have
getTimestamp() either, so I used beam.DoFn.TimestampParam to get the timestamp
twice, before and after Reshuffle. Assuming beam.DoFn.TimestampParam is always
the current timestamp bounded with an element, then it should work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 345714)
Time Spent: 10h 10m (was: 10h)
> Add more Python validates runner tests
> --------------------------------------
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
> Reporter: wendy liu
> Assignee: wendy liu
> Priority: Major
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
