[
https://issues.apache.org/jira/browse/BEAM-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16660769#comment-16660769
]
Thomas Weise commented on BEAM-5816:
------------------------------------
I changed the title to reflect the issue that needs to be looked into. We are
working around it in the portable runner by assuming that the dispose call can
fail because of the unexpected bundle processing.
> Flink runner starts new bundles while disposing operator
> ---------------------------------------------------------
>
> Key: BEAM-5816
> URL: https://issues.apache.org/jira/browse/BEAM-5816
> Project: Beam
> Issue Type: Bug
> Components: runner-flink
> Reporter: Micah Wylde
> Priority: Major
> Labels: portability-flink
>
> We sometimes see exceptions when shutting down portable flink pipelines
> (either due to cancellation or failure):
> {code}
> 2018-10-19 15:54:52,905 ERROR
> org.apache.flink.streaming.runtime.tasks.StreamTask - Error during
> disposal of stream operator.
> java.lang.RuntimeException: Failed to finish remote bundle
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.ExecutableStageDoFnOperator$SdkHarnessDoFnRunner.finishBundle(ExecutableStageDoFnOperator.java:241)
> at
> org.apache.beam.runners.flink.metrics.DoFnRunnerWithMetricsUpdate.finishBundle(DoFnRunnerWithMetricsUpdate.java:87)
> at
> org.apache.beam.runners.core.SimplePushbackSideInputDoFnRunner.finishBundle(SimplePushbackSideInputDoFnRunner.java:118)
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.invokeFinishBundle(DoFnOperator.java:674)
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.dispose(DoFnOperator.java:391)
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.ExecutableStageDoFnOperator.dispose(ExecutableStageDoFnOperator.java:166)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.disposeAllOperators(StreamTask.java:473)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:374)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:703)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.IllegalStateException: Already closed.
> at
> org.apache.beam.sdk.fn.data.BeamFnDataBufferingOutboundObserver.close(BeamFnDataBufferingOutboundObserver.java:95)
> at
> org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:251)
> at
> org.apache.beam.runners.flink.translation.wrappers.streaming.ExecutableStageDoFnOperator$SdkHarnessDoFnRunner.finishBundle(ExecutableStageDoFnOperator.java:238)
> ... 9 more
> Suppressed: java.lang.IllegalStateException: Processing bundle failed,
> TODO: [BEAM-3962] abort bundle.
> at
> org.apache.beam.runners.fnexecution.control.SdkHarnessClient$ActiveBundle.close(SdkHarnessClient.java:266)
> ... 10 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
