[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369585&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369585
]
ASF GitHub Bot logged work on BEAM-8575:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:06
Start Date: 10/Jan/20 03:06
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##########
File path: sdks/python/apache_beam/transforms/combiners_test.py
##########
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+ beam.CombineFn.__init__(self)
+ self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+ self.word_lengths_counter = Metrics.counter(
+ self.__class__, 'word_lengths')
+ self.word_lengths_dist = Metrics.distribution(
+ self.__class__, 'word_len_dist')
+ self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+ return ''
+
+ def add_input(self, acc, element):
+ self.word_counter.inc(1)
+ self.word_lengths_counter.inc(len(element))
+ self.word_lengths_dist.update(len(element))
+ self.last_word_len.set(len(element))
+
+ # ''.join() converts the list to a string.
+ return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
This is quite tricky.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed as ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])],
with modified code:
def create_accumulator(self):
return ''
def add_input(self, acc, element):
return acc + element
def merge_accumulators(self, accs):
return [ele for acc in accs for ele in acc]
def extract_output(self, acc):
return sorted(acc)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 369585)
Time Spent: 46h 50m (was: 46h 40m)
> Add more Python validates runner tests
> --------------------------------------
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
> Reporter: wendy liu
> Assignee: wendy liu
> Priority: Major
> Time Spent: 46h 50m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
