[
https://issues.apache.org/jira/browse/BEAM-6650?focusedWorklogId=199891&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199891
]
ASF GitHub Bot logged work on BEAM-6650:
----------------------------------------
Author: ASF GitHub Bot
Created on: 17/Feb/19 22:30
Start Date: 17/Feb/19 22:30
Worklog Time Spent: 10m
Work Description: mxm commented on issue #7810: [BEAM-6650] Add bundle
test with checkpointing for keyed processing
URL: https://github.com/apache/beam/pull/7810#issuecomment-464519090
@aljoscha I've replaced the `KeyGroupStateInternals` with
`SplitStateInternals`. Historically there might have been a reason for the two
ways of buffering elements during snapshotting, but as it stands now the split
state is the way to go. We don't gain anything by distributing the buffered
output elements onto key groups. The split state even leads to better
distribution of the buffered state, since it will be distributed round-robin
when restoring.
I think this is the best way to fix the leak in the snapshotting when
elements are emitted in `finalizeBundle`. Will follow-up with the
`prepareSnapshotPreBarrier` solution for 1.6/1.7, but it would be great if we
can get this in for the upcoming 2.11.0 release.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 199891)
Time Spent: 3h 50m (was: 3h 40m)
> FlinkRunner fails to checkpoint elements emitted during finishBundle
> --------------------------------------------------------------------
>
> Key: BEAM-6650
> URL: https://issues.apache.org/jira/browse/BEAM-6650
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Major
> Fix For: 2.11.0
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> Elements emitted during the finalizeBundle call in snapshopState are lost
> after the pipeline is restored. This only happens when the operator is keyed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
