[
https://issues.apache.org/jira/browse/BEAM-6796?focusedWorklogId=210898&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-210898
]
ASF GitHub Bot logged work on BEAM-6796:
----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Mar/19 09:27
Start Date: 11/Mar/19 09:27
Worklog Time Spent: 10m
Work Description: mxm commented on pull request #8023: [BEAM-6796][flink]
Fully finish bundle before timer callback
URL: https://github.com/apache/beam/pull/8023#discussion_r264145884
##########
File path:
runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/DoFnOperator.java
##########
@@ -679,6 +685,12 @@ protected final void invokeFinishBundle() {
pushbackDoFnRunner.finishBundle();
elementCount = 0L;
lastFinishBundleTime =
getProcessingTimeService().getCurrentProcessingTime();
+ // callback only after current bundle was fully finalized
+ // it could start a new bundle, for example resulting from timer
processing
+ if (bundleFinishedCallback != null) {
+ bundleFinishedCallback.run();
+ bundleFinishedCallback = null;
Review comment:
The stack trace makes perfect sense. The bundle is finalized (via the flag),
then started due to the watermark processing, then finalized immediately due to
the element count not being reset. The exception is thrown because there is no
active bundle. For the future, it would be helpful to share details directly in
the JIRA.
Due to the complexity of DoFnOperator I agree it is nicer to process the
callback there, instead of the subclass. I think we could add a test in a
follow-up to harden the semantics of the callback.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 210898)
Time Spent: 40m (was: 0.5h)
> Portable Flink runner error in finish bundle timer processing
> -------------------------------------------------------------
>
> Key: BEAM-6796
> URL: https://issues.apache.org/jira/browse/BEAM-6796
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Affects Versions: 2.11.0
> Reporter: Thomas Weise
> Assignee: Thomas Weise
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The timer callback fails since the bundle was already finished and a no new
> bundle started.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
