[
https://issues.apache.org/jira/browse/FLINK-11736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883502#comment-16883502
]
Jiangjie Qin edited comment on FLINK-11736 at 7/12/19 3:50 AM:
---------------------------------------------------------------
[~debugman] NOT_LEADER_FOR_PARTITION indicates that there was a leader
migration in the Kafka cluster for some partition. Such information of the new
leader will be propagated to all the brokers in the cluster asynchronously. It
may take some time, typically within 1 second, for all the brokers in the Kafka
cluster to receive the updated leader information.
>From the timestamp in the log, it seems the producer was retrying with a
>backoff around 3 ms. This may exhaust the 10 retries in 30 ms, which may be
>too aggressive. The default value of \{{retry.backoff.ms}} is actually 100ms.
>Did you happened to change that?
was (Author: becket_qin):
[~debugman] NOT_LEADER_FOR_PARTITION indicates that there was a leader
migration in the Kafka cluster for some partition. Such information of the new
leader will be propagated to all the brokers in the cluster asynchronously. It
may take some time, typically within 1 second, for all the brokers in the Kafka
cluster to receive the updated leader information.
>From the timestamp in the log, it seems the producer was retrying with a
>backoff around 3 ms. This may exhaust the 10 retries in 30 ms, which may be
>too aggressive. The default value is actually 100ms. Did you happened to
>change that?
> flink kafka producer failed with NOT_LEADER_FOR_PARTITION
> ---------------------------------------------------------
>
> Key: FLINK-11736
> URL: https://issues.apache.org/jira/browse/FLINK-11736
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Reporter: Kaicheng Sun
> Priority: Major
>
> my flink program connect kafka as its sinker using "FlinkKafkaProducer011"
> library. But sometimes the flink will exit abnormally with this kind of
> error:
> (2019-02-23 11:55:11,656 WARN
> org.apache.kafka.clients.producer.internals.Sender - Got error
> produce response with correlation id 55711 on topic-partition
> tmp_sink_redis-17, retrying (8 attempts left). Error: NOT_LEADER_FOR_PARTITION
> 2019-02-23 11:55:11,658 WARN
> org.apache.kafka.clients.producer.internals.Sender - Got error
> produce response with correlation id 55712 on topic-partition
> tmp_sink_redis-17, retrying (8 attempts left). Error:
> NOT_LEADER_FOR_PARTITION)
>
> The kafka cluster works properly, so I have no idea why this error will happen
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
