[
https://issues.apache.org/jira/browse/NIFI-2680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Moser updated NIFI-2680:
--------------------------------
Affects Version/s: (was: 0.8.0)
> PutKafka processors can fail but sill transfer the flowfile to the success
> output
> ---------------------------------------------------------------------------------
>
> Key: NIFI-2680
> URL: https://issues.apache.org/jira/browse/NIFI-2680
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.0.0, 0.7.0
> Reporter: Christopher McDermott
> Priority: Critical
> Fix For: 1.0.0, 0.8.0
>
>
>
> I have PutKafka processor where I’ve set the Max Record Size property to 1MB.
>
> If a transfer file larger than 1MiB to it, I see a bulletin and the following
> message in the log file, but the flowfile is transferred to the success
> output.
> This can lead to data loss in the flow.
>
> Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The
> message is 5800160 bytes when serialized which is larger than the maximum
> request size you have configured with the max.request.si\
> ze configuration.
> 2016-08-25 20:38:22,041 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.kafka.PutKafka
> PutKafka[id=3c396e30-7294-467e-b5d7-a3d44c0a04f4] Failed while waiting for
> acks from Kafka
> 2016-08-25 20:38:22,042 ERROR [Timer-Driven Process Thread-7]
> o.apache.nifi.processors.kafka.PutKafka
> java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.RecordTooLargeException: The message is
> 5800160 bytes when serialized which is larger than the maximum request size
> you have conf\
> igured with the max.request.size configuration.
> at
> org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:437)
> ~[kafka-clients-0.8.2.2.jar:na]
> at
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:352)
> ~[kafka-clients-0.8.2.2.jar:na]
> at
> org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:248)
> ~[kafka-clients-0.8.2.2.jar:na]
> at
> org.apache.nifi.processors.kafka.KafkaPublisher.publish(KafkaPublisher.java:137)
> ~[nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.processors.kafka.PutKafka$1.process(PutKafka.java:315)
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1822)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.processors.kafka.PutKafka.doRendezvousWithKafka(PutKafka.java:311)
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.processors.kafka.PutKafka.rendezvousWithKafka(PutKafka.java:287)
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.processors.kafka.AbstractKafkaProcessor.onTrigger(AbstractKafkaProcessor.java:76)
> [nifi-kafka-processors-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1054)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-oculus-0.7.x-SNAPSHOT.jar:oculus-0.7.x-SNAPSHOT]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_45]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_45]
>
> I have not yet tried the ProduceKafka processor
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
