[ 
https://issues.apache.org/jira/browse/NIFI-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16374485#comment-16374485
 ] 

Stephen Barry commented on NIFI-4616:
-------------------------------------

After conducting some further testing, I found that this is actually a config 
issue on Kafka broker side. Some background: I have a two node SSL-secured 
cluster and the replication factor of __consumer_offsets was the default 1. 
When one of my brokers died, there wasn't another replica for partitions which 
seemed to cause this hang and causes ConsumeKafka to stop consuming messages. 

I then increased the "offsets.topic.replication.factor" in server.properties 
and reassigned the existing __consumer_offsets partitions as per 
[https://stackoverflow.com/questions/46289511/kafka-reassignment-of-consumer-offsets-incorrect.]
 On re-running my tests, the issue no longer reproduces. NiFi ConsumeKafka 
consumes messages from topic even when one of my two brokers is down. Tested on 
ConsumeKafka_1_0 and ConsumeKafka_0_11.

> ConsumeKafka and ConsumeKafka_0_10 can block indefinitely if unable to 
> communicate with Kafka broker that is SSL enabled
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: NIFI-4616
>                 URL: https://issues.apache.org/jira/browse/NIFI-4616
>             Project: Apache NiFi
>          Issue Type: Bug
>          Components: Extensions
>    Affects Versions: 1.4.0
>            Reporter: Aldrin Piri
>            Priority: Major
>
> If I use ConsumeKafka and point to a broker that is in a bad state, I see 
> ConsumeKafka block indefinitely.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to