[
https://issues.apache.org/jira/browse/BEAM-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091872#comment-17091872
]
Stephen Patel edited comment on BEAM-9794 at 4/24/20, 8:10 PM:
---------------------------------------------------------------
Thanks [~mxm] and [~dmvk]!
was (Author: stephenpatel):
Thanks [~mxm]!
> Flink pipeline with RequiresStableInput fails after Short.MAX_VALUE
> checkpoints.
> --------------------------------------------------------------------------------
>
> Key: BEAM-9794
> URL: https://issues.apache.org/jira/browse/BEAM-9794
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.14.0, 2.15.0, 2.16.0, 2.17.0, 2.18.0, 2.19.0, 2.20.0
> Reporter: David Morávek
> Assignee: Maximilian Michels
> Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Full original report:
> https://lists.apache.org/thread.html/rb2ebfad16d85bcf668978b3defd442feda0903c20db29c323497a672%40%3Cuser.beam.apache.org%3E
> The exception comes from:
> https://github.com/apache/flink/blob/release-1.8.0/flink-runtime/src/main/java/org/apache/flink/runtime/state/OperatorBackendSerializationProxy.java#L68
> In the Flink Runner code, each checkpoint results in a new OperatorState (or
> KeyedState if the stream is keyed):
> https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L91-L103
> https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L136-L143
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
