Mark Payne created NIFI-2444:
--------------------------------
Summary: PublishKafka does not generate bulletins when there are
problems
Key: NIFI-2444
URL: https://issues.apache.org/jira/browse/NIFI-2444
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Mark Payne
Priority: Critical
Fix For: 1.0.0
I noticed extremely poor performance when trying to send data to Kafka via
PublishKafka. I saw no bulletins so didn't realize there were problems until I
looked into the logs and saw a lot of stack traces:
{code}
2016-08-01 00:30:03,749 ERROR [Timer-Driven Process Thread-44]
o.a.n.p.kafka.pubsub.KafkaPublisher Failed while waiting for acks from Kafka
java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to
access topics: [nifi-provenance]
at
org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:686)
~[kafka-clients-0.9.0.1.jar:na]
at
org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:449)
~[kafka-clients-0.9.0.1.jar:na]
at
org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:339)
~[kafka-clients-0.9.0.1.jar:na]
at
org.apache.nifi.processors.kafka.pubsub.KafkaPublisher.publish(KafkaPublisher.java:115)
~[nifi-kafka-pubsub-processors-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.processors.kafka.pubsub.PublishKafka$1.process(PublishKafka.java:248)
[nifi-kafka-pubsub-processors-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1880)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.processors.kafka.pubsub.PublishKafka.doRendezvousWithKafka(PublishKafka.java:244)
[nifi-kafka-pubsub-processors-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.processors.kafka.pubsub.PublishKafka.rendezvousWithKafka(PublishKafka.java:205)
[nifi-kafka-pubsub-processors-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.processors.kafka.pubsub.AbstractKafkaProcessor.onTrigger(AbstractKafkaProcessor.java:192)
[nifi-kafka-pubsub-processors-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_91]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_91]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_91]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_91]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_91]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
Caused by: org.apache.kafka.common.errors.TopicAuthorizationException: Not
authorized to access topics: [nifi-provenance]
{code}
It appears that the processor creates a KafkaPublisher class (and perhaps
others?) that have their own loggers. These loggers need to be removed and the
Processor's logger needs to be passed in so that all log messages generate the
appropriate bulletins.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
