[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=347122&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-347122
]
ASF GitHub Bot logged work on BEAM-8575:
----------------------------------------
Author: ASF GitHub Bot
Created on: 21/Nov/19 01:59
Start Date: 21/Nov/19 01:59
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10173: [BEAM-8575]
Added two unit tests in CombineTest class to test AccumulatingCombine
URL: https://github.com/apache/beam/pull/10173#discussion_r348863302
##########
File path: sdks/python/apache_beam/transforms/combiners_test.py
##########
@@ -393,6 +395,54 @@ def test_global_fanout(self):
| beam.CombineGlobally(combine.MeanCombineFn()).with_fanout(11))
assert_that(result, equal_to([49.5]))
+ @attr('ValidatesRunner')
+ def test_accumulating_combine(self):
+ with TestPipeline() as p:
+ input = (p
+ | beam.Create([('a', 1),
+ ('a', 1),
+ ('a', 4),
+ ('b', 1),
+ ('b', 13)]))
+ # The mean of all values regardless of key.
+ global_mean = (input
+ | beam.Values()
+ | beam.CombineGlobally(combine.MeanCombineFn()))
+
+ # The (key, mean) pairs for all keys.
+ mean_per_key = (input | beam.CombinePerKey(combine.MeanCombineFn()))
+
+ expected_mean_per_key = [('a', 2), ('b', 7)]
+ assert_that(global_mean, equal_to([4]), label='global mean')
+ assert_that(mean_per_key, equal_to(expected_mean_per_key),
+ label='mean per key')
+
+ @attr('ValidatesRunner')
+ def test_accumulating_combine_empty(self):
+ # For each element in a PCollection, if it is float('NaN'), then emits
+ # a string 'NaN', otherwise emits str(element).
+ class FormatNaNDoFn(beam.DoFn):
+ def process(self, element):
+ return ([str(element)], ['NaN'])[math.isnan(element)]
+
+ with TestPipeline() as p:
+ input = (p | beam.Create([]))
+
+ # Compute the mean of all values in the PCollection,
+ # then format the mean. Since the Pcollection is empty,
+ # the mean is float('NaN'), and is formatted to be a string 'NaN'.
+ global_mean = (input
+ | beam.Values()
+ | beam.CombineGlobally(combine.MeanCombineFn())
+ | beam.ParDo(FormatNaNDoFn()))
Review comment:
What about just doing beam.Map(str)?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 347122)
Time Spent: 15h 10m (was: 15h)
> Add more Python validates runner tests
> --------------------------------------
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
> Reporter: wendy liu
> Assignee: wendy liu
> Priority: Major
> Time Spent: 15h 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
