[
https://issues.apache.org/jira/browse/BEAM-11998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418754#comment-17418754
]
Jonathan Hourany edited comment on BEAM-11998 at 9/22/21, 6:23 PM:
-------------------------------------------------------------------
I'd like to make sure I understand the scope of this issue and that my own
ignorance isn't the actual problem. Does this ticket mean that there's no way
to read from Kafka in Python outside of the Dataflow, and by extention, GCP?
was (Author: jonathan hourany):
I'd like to make sure I understand the scope of this issue and that my own
newness to Beam isn't the culprit. Does this ticket mean that there's no way to
use `ReadFromKafka` in Python outside of the Dataflow, and by extention, GCP?
> Portable runners should be able to issue checkpoints to Splittable DoFn
> -----------------------------------------------------------------------
>
> Key: BEAM-11998
> URL: https://issues.apache.org/jira/browse/BEAM-11998
> Project: Beam
> Issue Type: New Feature
> Components: cross-language, runner-flink, runner-spark
> Reporter: Boyuan Zhang
> Priority: P2
> Attachments: read.png
>
>
> To execute unbounded Splittable DoFn over fnapi in streaming mode properly,
> portable runners should issue split(ProcessBundleSplitRequest with
> fraction_of_remainder > 0) or simply checkpoint(ProcessBundleSplitRequest
> with fraction_of_remainder == 0) to SDK regularly to make current bundle
> finished processing instead of running forever.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
