[jira] [Commented] (NIFI-12809) PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition
[ https://issues.apache.org/jira/browse/NIFI-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818102#comment-17818102 ] Steven Youtsey commented on NIFI-12809: --- I did not. That one seems to be spot on. > PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition > -- > > Key: NIFI-12809 > URL: https://issues.apache.org/jira/browse/NIFI-12809 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.25.0 >Reporter: Steven Youtsey >Priority: Major > Labels: kafka, partitioners, publish > > When configured to use the NiFi RoundRobin partitioner, the processor > publishes to every other partition. If the number of partitions in the topic > and the number of records being published are the right combination, this > problem is masked. We see this issue when we set the partitions to 26, but > not when set to 25. > I took a code-dive into the o.a.k.c.producer.KafkaProducer and discovered > that it is invoking the Partitioner twice when a "new batch" is created. > Thus, the RoundRobin partitioner bumps the index by 2. If the RoundRobin > partitioner overwrote the onNewBatch method, this problem could be solved. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-12809) PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition
[ https://issues.apache.org/jira/browse/NIFI-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818086#comment-17818086 ] Joe Witt commented on NIFI-12809: - [~slyouts] Did you review any Kafka bugs for the noted scenario? I see https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely related. > PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition > -- > > Key: NIFI-12809 > URL: https://issues.apache.org/jira/browse/NIFI-12809 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.25.0 >Reporter: Steven Youtsey >Priority: Major > Labels: kafka, partitioners, publish > > When configured to use the NiFi RoundRobin partitioner, the processor > publishes to every other partition. If the number of partitions in the topic > and the number of records being published are the right combination, this > problem is masked. We see this issue when we set the partitions to 26, but > not when set to 25. > I took a code-dive into the o.a.k.c.producer.KafkaProducer and discovered > that it is invoking the Partitioner twice when a "new batch" is created. > Thus, the RoundRobin partitioner bumps the index by 2. If the RoundRobin > partitioner overwrote the onNewBatch method, this problem could be solved. -- This message was sent by Atlassian Jira (v8.20.10#820010)
