[
https://issues.apache.org/jira/browse/BEAM-9487?focusedWorklogId=611498&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611498
]
ASF GitHub Bot logged work on BEAM-9487:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Jun/21 17:56
Start Date: 15/Jun/21 17:56
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #15003:
URL: https://github.com/apache/beam/pull/15003#discussion_r652024252
##########
File path: CHANGES.md
##########
@@ -103,10 +103,13 @@
it was ignored if provided.
* `TopCombineFn` disallow `compare` as its argument (Python)
([BEAM-7372](https://issues.apache.org/jira/browse/BEAM-7372)).
* Drop support for Flink 1.10
([BEAM-12281](https://issues.apache.org/jira/browse/BEAM-12281)).
+* Custom implementations of `TriggerFn` now require that `may_lose_data` be
implemented ([BEAM-9487](https://issues.apache.org/jira/browse/BEAM-9487)).
Review comment:
Same backward compatibility question applies. Would this break any
existing TriggerFn's out there.
And this is also a bit more severe then just adding a flag to workaround,
because it will require code changes. Often times that is very hard and
sometimes not possible. (E.g. An existing template where users do not have the
sources easily accessible.) This type of change might be better waiting for
Beam 3.0 (breaking changes) or happen very slowly.
##########
File path: sdks/python/apache_beam/transforms/ptransform_test.py
##########
@@ -476,6 +477,7 @@ def test_group_by_key(self):
result = pcoll | 'Group' >> beam.GroupByKey() | _SortLists
assert_that(result, equal_to([(1, [1, 2, 3]), (2, [1, 2]), (3, [1])]))
+ @skip('Behavior not available until Beam 2.33')
Review comment:
Can you set the flag manually here to avoid skipping?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 611498)
Time Spent: 15h 40m (was: 15.5h)
> GBKs on unbounded pcolls with global windows and no triggers should fail
> ------------------------------------------------------------------------
>
> Key: BEAM-9487
> URL: https://issues.apache.org/jira/browse/BEAM-9487
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Udi Meiri
> Assignee: Zachary Houfek
> Priority: P2
> Labels: EaseOfUse, starter
> Time Spent: 15h 40m
> Remaining Estimate: 0h
>
> This, according to "4.2.2.1 GroupByKey and unbounded PCollections" in
> https://beam.apache.org/documentation/programming-guide/.
> bq. If you do apply GroupByKey or CoGroupByKey to a group of unbounded
> PCollections without setting either a non-global windowing strategy, a
> trigger strategy, or both for each collection, Beam generates an
> IllegalStateException error at pipeline construction time.
> Example where this doesn't happen in Python SDK:
> https://stackoverflow.com/questions/60623246/merge-pcollection-with-apache-beam
> I also believe that this unit test should fail, since test_stream is
> unbounded, uses global window, and has no triggers.
> {code}
> def test_global_window_gbk_fail(self):
> with TestPipeline() as p:
> test_stream = TestStream()
> _ = p | test_stream | GroupByKey()
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
