[
https://issues.apache.org/jira/browse/BEAM-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17175705#comment-17175705
]
Boyuan Zhang commented on BEAM-6868:
------------------------------------
Thanks. Cham! There are 2 different errors mentioned on the
https://lists.apache.org/thread.html/re53d28276def9593dc5f28cbd9710d141aa0b68e2e0486d401724398%40%3Cdev.beam.apache.org%3E
thread:
* The ActiveBundle does not have a registered *bundle finalization handler*.
This means the SDK DoFn requests bundle finalization but the runner doesn't
support. In x-lang kafka read, it's required by the unbounded sdf wrapper.
* The ActiveBundle does not have a registered bundle *checkpoint handler* This
means the SDK DoFn returns a ProcessContinuation and may perform
self-checkpoint during execution time but the runner doesn't handle
DelayedBundleApplication. In x-lang kafka read, it's required by the sdf Kafka
implementation.
In order to use x-lang Kafka with flink runner, we can either support bundle
finalization in flink meanwhile use sdf wrapper for Kafka. Or we can support
self-checkpoint in flink and use sdf Kafka instead. But in long term, it would
be nice to have both in flink runner.
> Flink runner supports Bundle Finalization
> -----------------------------------------
>
> Key: BEAM-6868
> URL: https://issues.apache.org/jira/browse/BEAM-6868
> Project: Beam
> Issue Type: New Feature
> Components: cross-language, runner-flink
> Reporter: Boyuan Zhang
> Priority: P1
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
