[
https://issues.apache.org/jira/browse/BEAM-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228637#comment-15228637
]
ASF GitHub Bot commented on BEAM-162:
-------------------------------------
GitHub user mshields822 opened a pull request:
https://github.com/apache/incubator-beam/pull/135
[BEAM-162] Merging with closed windows less sensitive to bundle partioning
This change also rejects elements for windows which are closed before
merging.
(Rejecting elements which end up in a closed window post-merging is still
there.)
Idea is rejecting post-merge is necessary so that we never emit elements
for a closed window, and rejecting pre-merge is necessary so that the result is
invariant on bundle repartioning.
However:
- We want to abandon closable triggers.
- The notion of invariance on bundle repartioning is already unsupportable
for most of our triggers.
So I'd be fine if we give up on this one.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mshields822/incubator-beam beam-162
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/135.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #135
----
commit e1df0f2cdae123df301d2b8cebf42931852fe6b8
Author: Mark Shields <[email protected]>
Date: 2016-04-05T22:12:16Z
Formatting fiddles
commit 65874af3abf35eed66976b47f9712eec4e06304c
Author: Mark Shields <[email protected]>
Date: 2016-04-06T04:20:06Z
Also reject closed pre-merged.
----
> assert fail using session windows
> ---------------------------------
>
> Key: BEAM-162
> URL: https://issues.apache.org/jira/browse/BEAM-162
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Mark Shields
> Assignee: Mark Shields
>
> java.lang.IllegalStateException: Window
> [2016-03-31T05:35:31.158Z..2016-03-31T06:05:31.158Z) should have been added
> at
> com.google.cloud.dataflow.sdk.repackaged.com.google.common.base.Preconditions.checkState(Preconditions.java:199)
> at
> com.google.cloud.dataflow.sdk.util.ReduceFnRunner.processElement(ReduceFnRunner.java:440)
> at
> com.google.cloud.dataflow.sdk.util.ReduceFnRunner.processElements(ReduceFnRunner.java:282)
> at
> com.google.cloud.dataflow.sdk.util.GroupAlsoByWindowViaWindowSetDoFn.processElement(GroupAlsoByWindowViaWindowSetDoFn.java:83)
> at
> com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49)
> at
> com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:138)
> at
> com.google.cloud.dataflow.sdk.util.LateDataDroppingDoFnRunner.processElement(LateDataDroppingDoFnRunner.java:67)
> at
> com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:191)
> at
> com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42)
> at
> com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47)
> at
> com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:53)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
