[ https://issues.apache.org/jira/browse/NIFI-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818086#comment-17818086 ]
Joe Witt edited comment on NIFI-12809 at 2/16/24 7:56 PM: ---------------------------------------------------------- [~slyouts] Did you review any Kafka bugs for the noted scenario? I see https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely related. If there is a reasonable workaround for this issue that we can control on the NiFi side we should ensure we remove any such change in versions it is resolved in (such as Kafka 3 components/etc..) was (Author: joewitt): [~slyouts] Did you review any Kafka bugs for the noted scenario? I see https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely related. > PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition > ------------------------------------------------------------------------------ > > Key: NIFI-12809 > URL: https://issues.apache.org/jira/browse/NIFI-12809 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Affects Versions: 1.25.0 > Reporter: Steven Youtsey > Priority: Major > Labels: kafka, partitioners, publish > > When configured to use the NiFi RoundRobin partitioner, the processor > publishes to every other partition. If the number of partitions in the topic > and the number of records being published are the right combination, this > problem is masked. We see this issue when we set the partitions to 26, but > not when set to 25. > I took a code-dive into the o.a.k.c.producer.KafkaProducer and discovered > that it is invoking the Partitioner twice when a "new batch" is created. > Thus, the RoundRobin partitioner bumps the index by 2. If the RoundRobin > partitioner overwrote the onNewBatch method, this problem could be solved. -- This message was sent by Atlassian Jira (v8.20.10#820010)