[
https://issues.apache.org/jira/browse/BEAM-6733?focusedWorklogId=348533&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-348533
]
ASF GitHub Bot logged work on BEAM-6733:
----------------------------------------
Author: ASF GitHub Bot
Created on: 23/Nov/19 15:46
Start Date: 23/Nov/19 15:46
Worklog Time Spent: 10m
Work Description: stale[bot] commented on issue #9652: [BEAM-6733] Use
Flink's prepareSnapshotPreBarrier to replace BufferedOutputManager
URL: https://github.com/apache/beam/pull/9652#issuecomment-557808518
This pull request has been marked as stale due to 60 days of inactivity. It
will be closed in 1 week if no further activity occurs. If you think that’s
incorrect or this pull request requires a review, please simply write any
comment. If closed, you can revive the PR at any time and @mention a reviewer
or discuss it on the [email protected] list. Thank you for your
contributions.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 348533)
Time Spent: 2h 40m (was: 2.5h)
> Use Flink 1.6/1.7 prepareSnapshotPreBarrier to replace BufferedOutputManager
> ----------------------------------------------------------------------------
>
> Key: BEAM-6733
> URL: https://issues.apache.org/jira/browse/BEAM-6733
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Reporter: Maximilian Michels
> Assignee: Maximilian Michels
> Priority: Minor
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Flink 1.6/1.7 provides a hook to execute an action before the snapshot
> barrier is emitted by the operator. At the moment (<=1.5) the Flink Runner
> has to buffer any elements which are emitted during a snapshot because the
> barrier has already been emitted. This leads to a lot of code complexity.
> We can remove the buffering in favor of finishing the current bundle in
> {{DoFnOperator}}'s {{prepareSnapshotPreBarrier}}. The 1.5/1.6/1.7 build setup
> poses a challenge to do that in a way that does not lead to much code
> duplication.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
