[ https://issues.apache.org/jira/browse/BEAM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860992#comment-15860992 ]
Jingsong Lee commented on BEAM-1393: ------------------------------------ Good point! The processing of pushed-back events is indeed a trouble. For non-keyed operators, we store the elements in SPLIT_DISTRIBUTE state, this is no problem. But for keyed operators, we can't find the prepared events when a new side-input element come if we use {{KeyedStateBackend}}. We need to find all the pushed-back events that have the side-input window. Just like the processing of timer. Maybe we need override {{AbstractStreamOperator.snapshotState}} to store pushed-back events by KeyGroups way with snapshot TimerService. I see that only one {{startNewKeyGroup}} can be called, so we have to override the TimerService snapshot instead of calling super. > Update Flink Runner to Flink 1.2.0 > ---------------------------------- > > Key: BEAM-1393 > URL: https://issues.apache.org/jira/browse/BEAM-1393 > Project: Beam > Issue Type: Improvement > Components: runner-flink > Reporter: Aljoscha Krettek > Assignee: Jingsong Lee > > When we update to 1.2.0 we can use the new internal Timer API that is > available to Flink operators: {{InternalTimerService}} and also use broadcast > state to store side-input data. -- This message was sent by Atlassian JIRA (v6.3.15#6346)