[
https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860992#comment-15860992
]
Jingsong Lee commented on BEAM-1393:
------------------------------------
Good point! The processing of pushed-back events is indeed a trouble. For
non-keyed operators, we store the elements in SPLIT_DISTRIBUTE state, this is
no problem. But for keyed operators, we can't find the prepared events when a
new side-input element come if we use {{KeyedStateBackend}}. We need to find
all the pushed-back events that have the side-input window. Just like the
processing of timer.
Maybe we need override {{AbstractStreamOperator.snapshotState}} to store
pushed-back events by KeyGroups way with snapshot TimerService. I see that only
one {{startNewKeyGroup}} can be called, so we have to override the TimerService
snapshot instead of calling super.
> Update Flink Runner to Flink 1.2.0
> ----------------------------------
>
> Key: BEAM-1393
> URL: https://issues.apache.org/jira/browse/BEAM-1393
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Aljoscha Krettek
> Assignee: Jingsong Lee
>
> When we update to 1.2.0 we can use the new internal Timer API that is
> available to Flink operators: {{InternalTimerService}} and also use broadcast
> state to store side-input data.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
