[
https://issues.apache.org/jira/browse/FLINK-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024043#comment-15024043
]
Robert Metzger commented on FLINK-3066:
---------------------------------------
Hi Gyula,
this behavior has been implemented like this by design. We didn't want to
re-implement the Kafka high level consumer while the Kafka project is cooking
up something much better.
The leader change is something we didn't implement in our consumer.
How often do you have leader changes in your environment? Is this a critical
issue for using Flink in production?
> Kafka source fails on leader change
> -----------------------------------
>
> Key: FLINK-3066
> URL: https://issues.apache.org/jira/browse/FLINK-3066
> Project: Flink
> Issue Type: Bug
> Components: Streaming Connectors
> Affects Versions: 0.10.0, 1.0.0
> Reporter: Gyula Fora
>
> I got the following exception during my streaming job:
> 16:44:50,637 INFO org.apache.flink.runtime.jobmanager.JobManager
> - Status of job 4d3f9443df4822e875f1400244a6e8dd (deduplo!) changed to
> FAILING.
> java.lang.Exception: Failed to send data to Kafka: This server is not the
> leader for that topic-partition.
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.checkErroneous(FlinkKafkaProducer.java:275)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.invoke(FlinkKafkaProducer.java:246)
> at
> org.apache.flink.streaming.api.operators.StreamSink.processElement(StreamSink.java:37)
> at
> org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:166)
> at
> org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:63)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:221)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:588)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.errors.NotLeaderForPartitionException:
> This server is not the leader for that topic-partition.
> And then the job crashed and recovered. This should probably be something
> that we handle without crashing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
