[
https://issues.apache.org/jira/browse/NIFI-10077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard resolved NIFI-10077.
-----------------------------------
Resolution: Feedback Received
Apache NiFi 1.x is no longer maintained and no new release is planned on the
1.x release line. Marking as resolved as part of a cleanup operation. Please
open a new one with an updated description if this is still relevant for NiFi
2.x.
> ConsumeKafkaRecord_2_6 not consuming data when using patterns for topic names
> -----------------------------------------------------------------------------
>
> Key: NIFI-10077
> URL: https://issues.apache.org/jira/browse/NIFI-10077
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.16.2
> Environment: CentOS 7.9
> OpenJDK 11.0.10
> Kafka 2.3.x
> 9 brokers (in the environment I'm testing)
> 18 topics match the regex pattern
> 156 partitions total. Processor has 30 threads.
> Reporter: Corey Shaw
> Priority: Major
>
> I've been seeing some odd behavior with the ConsumeKafkaRecord_2_6 processor
> (on NiFi 1.16.2). I have been testing out using "Topic Name Format" set to
> "pattern". I've found that no matter what I set for the "Topic Name(s)"
> value, the processor never actually starts consuming data. It will properly
> talk to Kafka and join the necessary consumer group, but data is never read.
>
> I have confirmed in both the NiFi logs and Kafka itself (using the
> "kafka-consumer-groups" CLI tool) that the processor has in fact connected
> and joined the consumer group. If I instead use a "Topic Name Format" set to
> "names" (and "Topic Name(s)" changed accordingly to a comma-delimited list),
> then everything functions perfectly (data is consumed as it should be).
>
> h2. Example One (Works)
> "Topic Name Format" set to "names" and "Topic Name(s)" set to
> "topic1,topic2" :
> * Processor connects to Kafka fine
> * Processor joins the consumer group fine and shows as consuming from the
> expected topics
> * Processor consumes data from both topics as expected
> h2. Example Two (Works)
> "Topic Name Format" set to "names" and "Topic Name(s)" set to "topic1":
> * Processor connects to Kafka fine
> * Processor joins the consumer group fine and shows as consuming from the
> expected topic
> * Processor consumes data from the topic as expected
> h2. Example Three (Fails)
> "Topic Name Format" set to "pattern" and "Topic Name(s)" set to "topic\d":
> * Processor connects to Kafka fine
> * Processor joins the consumer group fine and shows as consuming from the
> expected topics
> * No data is consumed by the processor
> h2. Example Four (Fails)
> "Topic Name Format" set to "pattern" and "Topic Name(s)" set to "topic1":
> * Processor connects to Kafka fine
> * Processor joins the consumer group fine and shows as consuming from the
> expected topics
> * No data is consumed by the processor
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
