Thomas Groh created BEAM-2259:
---------------------------------
Summary: Reshuffle may set watermark holds past the end of time
Key: BEAM-2259
URL: https://issues.apache.org/jira/browse/BEAM-2259
Project: Beam
Issue Type: Bug
Components: sdk-java-core
Reporter: Thomas Groh
Assignee: Thomas Groh
When it does so, the ReduceFnRunner throws an exception that the watermark hold
is past the end of time
This affects all non-Dataflow runners.
(With additional checks)
Caused by: java.lang.IllegalArgumentException: Assigned
TimestampedValueInSingleWindow{value=KV{key: gameexamples:test_tgroh.l
eaderboard_team shard: 33,
org.apache.beam.sdk.io.gcp.bigquery.TableRowInfo@31436a52},
timestamp=2017-05-11T15:29:59.999Z, window=[20
17-05-11T15:15:00.000Z..2017-05-11T15:30:00.000Z), pane=PaneInfo{isFirst=true,
timing=EARLY, index=0}} to a window [2017-05-11T15:15:
00.000Z..2017-05-11T15:30:00.000Z) with fn
org.apache.beam.sdk.util.IdentityWindowFn@58543f9f. Expiry is past end of time.
Windowing
strategy
WindowingStrategy{windowFn=org.apache.beam.sdk.util.IdentityWindowFn@58543f9f,
allowedLateness=PT9223372036854.775S, trigger
=ReshuffleTrigger(), accumulationMode=DISCARDING_FIRED_PANES,
timestampCombiner=EARLIEST}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
