[
https://issues.apache.org/jira/browse/NIFI-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bartłomiej Tartanus updated NIFI-6836:
--------------------------------------
Description:
ConsumeKafkaRecord_1_0 reads data from first 100 topics and ignores the rest.
This bug is caused by this line:
[https://github.com/apache/nifi/blob/45ebeba846cf257c0dd27669c25e244208758fb9/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-1-0-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumeKafkaRecord_1_0.java#L316]
This might be easily fixed by removing {{,100}} but I assume someone put this
limit for a reason. If this limit is needed then some mention in documentation
should be added. Also a validator that checks if number of topics is not
greater than 100.
I can prepare merge request if we agree what to do with this issue.
Current workaround is to add another ConsumeKafkaRecord_1_0 processor and split
the list in parts, each containing at most 100 topics.
was:
ConsumeKafkaRecord_1_0 reads data from first 100 topics and ignores the rest.
This bug is caused by this line:
[https://github.com/apache/nifi/blob/45ebeba846cf257c0dd27669c25e244208758fb9/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-1-0-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumeKafkaRecord_1_0.java#L316]
This might be easily fixed by removing `,100` but I assume someone put this
limit for a reason. If this limit is needed then some mention in documentation
should be added. Also a validator that checks if number of topics is not
greater than 100.
I can prepare merge request if we agree what to do with this issue.
Current workaround is to add another ConsumeKafkaRecord_1_0 processor and split
the list in parts, each containing at most 100 topics.
> ConsumeKafkaRecord_1_0 reads data from first 100 topics
> -------------------------------------------------------
>
> Key: NIFI-6836
> URL: https://issues.apache.org/jira/browse/NIFI-6836
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.9.2
> Reporter: Bartłomiej Tartanus
> Priority: Major
> Labels: easyfix
>
> ConsumeKafkaRecord_1_0 reads data from first 100 topics and ignores the rest.
> This bug is caused by this line:
>
> [https://github.com/apache/nifi/blob/45ebeba846cf257c0dd27669c25e244208758fb9/nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-1-0-processors/src/main/java/org/apache/nifi/processors/kafka/pubsub/ConsumeKafkaRecord_1_0.java#L316]
> This might be easily fixed by removing {{,100}} but I assume someone put this
> limit for a reason. If this limit is needed then some mention in
> documentation should be added. Also a validator that checks if number of
> topics is not greater than 100.
> I can prepare merge request if we agree what to do with this issue.
>
> Current workaround is to add another ConsumeKafkaRecord_1_0 processor and
> split the list in parts, each containing at most 100 topics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
