Maximilian Michels created BEAM-6751:
----------------------------------------
Summary: KafkaIO blocks FlinkRunner in EOS mode
Key: BEAM-6751
URL: https://issues.apache.org/jira/browse/BEAM-6751
Project: Beam
Issue Type: Bug
Components: io-java-kafka, runner-flink
Reporter: Maximilian Michels
Assignee: Maximilian Michels
Fix For: 2.12.0
KafkaIO has a validation check which whitelists certain runners capable of
provide exactly-once semantics:
{noformat}
if ("org.apache.beam.runners.direct.DirectRunner".equals(runner)
|| runner.startsWith("org.apache.beam.runners.dataflow.")
|| runner.startsWith("org.apache.beam.runners.spark.") {
...
{noformat}
The FlinkRunner supports exactly-once checkpointing but is blocked from using
Kafka's exactly once mode.
I wonder if such a list is easily maintainable? I think we should replace the
list with a warning instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
