[
https://issues.apache.org/jira/browse/BEAM-10558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maximilian Michels reopened BEAM-10558:
---------------------------------------
> Flushing of buffered elements during checkpoint can stall
> ---------------------------------------------------------
>
> Key: BEAM-10558
> URL: https://issues.apache.org/jira/browse/BEAM-10558
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: P2
> Fix For: 2.24.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Elements are buffered during {{DoFnOperator#snapshotState}}, called as part
> of a Flink checkpoint. This is necessary because flushing out elements in
> this method call would alter the checkpoint barrier alignment. Optionally,
> elements can be flushed out before the method call via the
> {{finishBundleBeforeCheckpointing}} option which is turned off by default
> because it can affect the checkpoint duration.
> The buffer is flushed as part of starting a new bundle. A problem arises if
> no new bundle will be started. For example, this can be the case if only a
> single element (e.g. Impulse) is produced as part of a bundle during
> checkpointing. Afterwards, when no bundle will be started due to another
> element arriving, or a timer firing, the element will not be flushed from the
> buffer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
