[
https://issues.apache.org/jira/browse/NIFI-12809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17818086#comment-17818086
]
Joe Witt edited comment on NIFI-12809 at 2/16/24 7:56 PM:
----------------------------------------------------------
[~slyouts] Did you review any Kafka bugs for the noted scenario? I see
https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely
related.
If there is a reasonable workaround for this issue that we can control on the
NiFi side we should ensure we remove any such change in versions it is resolved
in (such as Kafka 3 components/etc..)
was (Author: joewitt):
[~slyouts] Did you review any Kafka bugs for the noted scenario? I see
https://issues.apache.org/jira/browse/KAFKA-13180 that sounds very closely
related.
> PublishKafkaRecord_2_6 - RoundRobin partitioner skipping every other partition
> ------------------------------------------------------------------------------
>
> Key: NIFI-12809
> URL: https://issues.apache.org/jira/browse/NIFI-12809
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.25.0
> Reporter: Steven Youtsey
> Priority: Major
> Labels: kafka, partitioners, publish
>
> When configured to use the NiFi RoundRobin partitioner, the processor
> publishes to every other partition. If the number of partitions in the topic
> and the number of records being published are the right combination, this
> problem is masked. We see this issue when we set the partitions to 26, but
> not when set to 25.
> I took a code-dive into the o.a.k.c.producer.KafkaProducer and discovered
> that it is invoking the Partitioner twice when a "new batch" is created.
> Thus, the RoundRobin partitioner bumps the index by 2. If the RoundRobin
> partitioner overwrote the onNewBatch method, this problem could be solved.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
