Maximilian Michels created BEAM-6733:
----------------------------------------
Summary: Use Flink 1.6/1.7 prepareSnapshotPreBarrier to replace
BufferedOutputManager
Key: BEAM-6733
URL: https://issues.apache.org/jira/browse/BEAM-6733
Project: Beam
Issue Type: Improvement
Components: runner-flink
Reporter: Maximilian Michels
Assignee: Maximilian Michels
Flink 1.6/1.7 provides a hook to execute an action before the snapshot barrier
is emitted by the operator. At the moment (<=1.5) the Flink Runner has to
buffer any elements which are emitted during a snapshot because the barrier has
already been emitted. This leads to a lot of code complexity.
We can remove the buffering in favor of finishing the current bundle in
{{DoFnOperator}}'s {{prepareSnapshotPreBarrier}}. The 1.5/1.6/1.7 build setup
poses a challenge to do that in a way that does not lead to much code
duplication.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
