[
https://issues.apache.org/jira/browse/BEAM-10691?focusedWorklogId=470146&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-470146
]
ASF GitHub Bot logged work on BEAM-10691:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Aug/20 08:52
Start Date: 13/Aug/20 08:52
Worklog Time Spent: 10m
Work Description: je-ik commented on pull request #12551:
URL: https://github.com/apache/beam/pull/12551#issuecomment-673351917
Another observation is that this happens when there are many timers that are
fired at the same time. The pipeline operates in bootstrap mode - watermark is
updated after reading 1 hour of data from batch storage, so it "hops" on 1 hour
boundaries, but the pipeline uses 30s windows. So there are many windows that
get closed and that's where this happens. It seems no to happen when the
pipeline reads realtime data and updates watermark appropriately. I think there
might be some sort of race condition somewhere, but so far I didn't figure out
where exactly.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 470146)
Time Spent: 2h 10m (was: 2h)
> FlinkRunner: pipeline might get stuck due to timer watermark hold not being
> released
> ------------------------------------------------------------------------------------
>
> Key: BEAM-10691
> URL: https://issues.apache.org/jira/browse/BEAM-10691
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.23.0, 2.24.0
> Reporter: Jan Lukavský
> Assignee: Jan Lukavský
> Priority: P1
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Pipeline might stop progressing watermark in certain cases due to timer
> output timestamp not being released from
> FlinkTimerInternals#outputTimestampQueue. The pipeline has to be restarted
> from checkpoint to reload the cache and free watermark hold.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
