[
https://issues.apache.org/jira/browse/BEAM-14499?focusedWorklogId=773677&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-773677
]
ASF GitHub Bot logged work on BEAM-14499:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 23/May/22 19:12
Start Date: 23/May/22 19:12
Worklog Time Spent: 10m
Work Description: jrmccluskey commented on code in PR #17733:
URL: https://github.com/apache/beam/pull/17733#discussion_r879790460
##########
sdks/go/pkg/beam/testing/passert/equals.go:
##########
@@ -53,13 +54,37 @@ func EqualsList(s beam.Scope, col beam.PCollection, list
interface{}) beam.PColl
return equals(subScope, col, listCollection)
}
+// WindowedEqualsList verifies that the given collection has the same values
as a
+// given list, under coder equality. The values must be provided as an array
or a slice.
+// This function also takes a window function to window the list into for
cases where
+// the PCollections cannot be globally windowed (e.g. tests in unbounded
pipelines.)
+// This windowing function is applied to both the PCollection created from the
list
+// and the impulse used to trigger the Diff function.
+func WindowedEqualsList(s beam.Scope, wfn *window.Fn, col beam.PCollection,
list interface{}) beam.PCollection {
Review Comment:
Oh I see, you want `Equals()` and `EqualsList()` to actually window into a
big interval instead of using the global window. I'm not a fan of introducing
windowing when it isn't being used explicitly by the user, personally. It isn't
intuitive to be evaluating PCollection equality in that way in a global batch
context.
We've also found that the current test set-up actually creates a very
brittle set-up with a windowed impulse only firing once and not per-window, so
this approach doesn't work correctly. PeriodicImpulse and streaming-specific
test utilities are really needed.
Issue Time Tracking
-------------------
Worklog Id: (was: 773677)
Time Spent: 3h 20m (was: 3h 10m)
> TestStream tests are broken after new construction check was added
> ------------------------------------------------------------------
>
> Key: BEAM-14499
> URL: https://issues.apache.org/jira/browse/BEAM-14499
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Jack McCluskey
> Assignee: Jack McCluskey
> Priority: P2
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The Go SDK's TestStream integration tests are currently failing as a result
> of the restrictions on creating unwindowed, unbounded side inputs being added
> for BEAM-14473. These should be fixed so the Flink PostCommit tests give
> signal again.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
