Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/1906
I tested with following schema:
```json
{
"type": "record", "name": "test",
"fields": [
{ "name": "id", "type": "long" },
{ "name": "name", "type": [ "null", "string" ] }
]
}
```
And this message:
```json
{"name": "does not have ID"}
```
Then I got following exception, the message was not routed to 'failure':
```
2017-06-12 14:46:26,967 ERROR [Timer-Driven Process Thread-10]
o.a.n.p.k.pubsub.ConsumeKafkaRecord_0_10
ConsumeKafkaRecord_0_10[id=9ad0582e-015c-1000-3b99-4fb95ea47d32] Exception
while processing data from kafka so will close the lease
org.apache.nifi.processors.kafka.pubsub.ConsumerPool$SimpleConsumerLease@2b2cc16c
due to org.apache.nifi.processor.exception.ProcessException:
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of long in field id of test:
org.apache.nifi.processor.exception.ProcessException:
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of long in field id of
testorg.apache.nifi.processor.exception.ProcessException:
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of long in field id of test
at
org.apache.nifi.processors.kafka.pubsub.ConsumerLease.writeRecordData(ConsumerLease.java:528)
at
org.apache.nifi.processors.kafka.pubsub.ConsumerLease.lambda$processRecords$2(ConsumerLease.java:320)
at
java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1548)
at
java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
at
org.apache.nifi.processors.kafka.pubsub.ConsumerLease.processRecords(ConsumerLease.java:307)
at
org.apache.nifi.processors.kafka.pubsub.ConsumerLease.poll(ConsumerLease.java:168)
at
org.apache.nifi.processors.kafka.pubsub.ConsumeKafkaRecord_0_10.onTrigger(ConsumeKafkaRecord_0_10.java:327)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of long in field id of test
at
org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at
org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:59)
at
org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
at
org.apache.nifi.processors.kafka.pubsub.ConsumerLease.writeRecordData(ConsumerLease.java:506)
... 18 common frames omitted
Caused by: java.lang.NullPointerException: null of long in field id of test
at
org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:132)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:126)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
at
org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 21 common frames omitted
Caused by: java.lang.NullPointerException: null
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
... 24 common frames omitted
```
Probably we need to wrap `writer.write(record)` as well?
https://github.com/apache/nifi/pull/1906/files#diff-5a7aa2af019388ff0cd5be33b8fbd660R506
There is the template file that I used.
https://gist.github.com/ijokarumawak/b5943f83e291f4c08adec9cf4add2e26
@markap14 Can you take a look?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
