Koji Kawamura created NIFI-3363:
-----------------------------------
Summary: Putkafka processor throws null pointer exception when we
choose User-Defined strategy
Key: NIFI-3363
URL: https://issues.apache.org/jira/browse/NIFI-3363
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 1.0.1, 1.1.1, 0.7.1, 1.1.0, 0.6.1, 0.7.0, 0.5.1, 0.4.1,
0.6.0, 0.5.0, 0.4.0, 1.0.0
Environment: By looking at the release tags contained in a commit that
added User-Defined partition strategy (NIFI-1097 22de23b), it seems all NiFi
versions since 0.4.0 is affected.
Reporter: Koji Kawamura
Assignee: Koji Kawamura
NullPointerException is thrown because PutKafka tries to put null into
properties since following if statements doesn't cover
USER_DEFINED_PARTITIONING.
{code: title=PutKafka.java buildKafkaConfigProperties}
String partitionStrategy = context.getProperty(PARTITION_STRATEGY).getValue();
String partitionerClass = null;
if (partitionStrategy.equalsIgnoreCase(ROUND_ROBIN_PARTITIONING.getValue())) {
partitionerClass = Partitioners.RoundRobinPartitioner.class.getName();
} else if (partitionStrategy.equalsIgnoreCase(RANDOM_PARTITIONING.getValue())) {
partitionerClass = Partitioners.RandomPartitioner.class.getName();
}
properties.setProperty("partitioner.class", partitionerClass); // Happens here
{code}
A naive fix for this would be adding one more if statement so that it puts
'partitioner.class' property only if partitionerClass is set.
However, while I was testing the fix, I found following facts, that revealed
this approach wouldn't be the right solution for this issue.
In short, we don't have to set 'partitioner.class' property with Kafka 0.8.x
client in the first place. I assume it's there because PutKafka came through a
long history..
h2. PutKafka history analysis
- PutKafka used to cover Kafka 0.8 and 0.9
- Around the time Kafka 0.9 was released, PutKafka added 'partitioner.class'
via NIFI-1097: start using new API. There were two client libraries
kafka-clients and kafka_2.9.1, both 0.8.2.2.
- Then PublishKafka is added for Kafka 0.9, at this point, we could add
'partition' property to PublishKafka, but we didn't do that for some reason.
PublishKafka doesn't support user defined partition as of this writing.
NIFI-1296.
- The code adding 'partitioner.class' has been left in PutKafka.
- Further, we separated nar into 0.8, 0.9 and 0.10.
- Now only PutKafka(0.8) uses 'partitioner.class' property, but 0.8 client
doesn't use that property. So we don't need that code at all.
h2. Then, how should we fix this?
Since PutKafka in both master and 0.x branch specifically uses Kafka 0.8.x
client. We can simply remove the codes adding 'partitioner.class', probably
PARTITION_STRATEGY processor property, too.
h2. Expected result after fix
- Users can specify Kafka partition with PutKafka 'partition' property, but no
need to specify 'partition strategy', NullPointerException won't be thrown
- A warning log, that used to be logged in nifi-app.log won't be logged any
more: {code}2017-01-18 13:53:33,071 WARN [Timer-Driven Process Thread-9]
o.a.k.clients.producer.ProducerConfig The configuration partitioner.class =
null was supplied but isn't a known config.{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
