[
https://issues.apache.org/jira/browse/BEAM-11143?focusedWorklogId=505418&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-505418
]
ASF GitHub Bot logged work on BEAM-11143:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 27/Oct/20 21:34
Start Date: 27/Oct/20 21:34
Worklog Time Spent: 10m
Work Description: scwhittle commented on pull request #13206:
URL: https://github.com/apache/beam/pull/13206#issuecomment-717553720
R: @kennknowles
This was impacting a dataflow customer. As I was debugging I also found
https://issues.apache.org/jira/browse/BEAM-11144 but I plan on doing that
separately. Both of these issues show how the contexts and the namespace used
can be confusing. To debug I made changes to ReduceFnRunner to validate that
the in-memory state was all empty at the end of certain tests. Let me know if
you'd like me to include that here. It would be nice to have support in
ValidateRunner tests that it didn't leak state, does that exist or do you have
any ideas?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 505418)
Time Spent: 20m (was: 10m)
> AfterWatermark can leak state for the late trigger when merging windows
> -----------------------------------------------------------------------
>
> Key: BEAM-11143
> URL: https://issues.apache.org/jira/browse/BEAM-11143
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Sam Whittle
> Assignee: Sam Whittle
> Priority: P2
> Time Spent: 20m
> Remaining Estimate: 0h
>
> AfterWatermarkStateMachine attempts to clear the late trigger state when
> merging and the early trigger is not closed in some windows. However it
> appears to clear the state corresponding to the new window, not all of the
> states for the input windows. This leaks the state which can lead to
> unbounded pipeline state buildup.
> https://github.com/apache/beam/blob/68d6c8e6243b1d8f392840273f886276e2a8baff/runners/core-java/src/main/java/org/apache/beam/runners/core/triggers/AfterWatermarkStateMachine.java#L133
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
