[
https://issues.apache.org/jira/browse/NIFI-4030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16041324#comment-16041324
]
ASF GitHub Bot commented on NIFI-4030:
--------------------------------------
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/1896
@bbende looks good. +1 merged to master
> Avro Schema with default values can cause NPE
> ---------------------------------------------
>
> Key: NIFI-4030
> URL: https://issues.apache.org/jira/browse/NIFI-4030
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.2.0, 1.3.0
> Reporter: Bryan Bende
> Assignee: Bryan Bende
> Priority: Minor
> Fix For: 1.4.0
>
> Attachments: Convert_Avro_Short_to_Full.xml
>
>
> While coming up with an example to convert between Avro schemas, I ran into
> the following scenario...
> I created two schemas like the following:
> {code}
> {
> "name": "shortSchema",
> "namespace": "nifi",
> "type": "record",
> "fields": [
> { "name": "a", "type": "string" },
> { "name": "b", "type": "string" }
> ]
> }
> {code}
> {code}
> {
> "name": "fullSchema",
> "namespace": "nifi",
> "type": "record",
> "fields": [
> { "name": "c", "type": "string", "default" : "default value for field c"
> },
> { "name": "d", "type": "string", "default" : "default value for field d"
> },
> { "name": "a", "type": "string" },
> { "name": "b", "type": "string" }
> ]
> }
> {code}
> Then I created a flow with a ConvertRecord processor that read a flow file
> which had Avro created with the shortSchema, and used a writer configured to
> write with the fullSchema.
> This produced the following exception, even though the fullSchema has default
> values for additional fields:
> {code}
> ERROR [Timer-Driven Process Thread-1] o.a.n.processors.standard.ConvertRecord
> ConvertRecord[id=ec373b3a-5704-1c6b-f501-0977a332d513] Failed to process
> StandardFlowFileRecord[uuid=a8e604a9-1e15-4b9b-a4b7-fdbd441d3f2a,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1496842800576-10, container=default,
> section=10], offset=40141,
> length=196],offset=0,name=128518448076937,size=196]:
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> java.lang.NullPointerException: null of string in field c of nifi.fullSchema
> org.apache.avro.file.DataFileWriter$AppendWriteException:
> java.lang.NullPointerException: null of string in field c of nifi.fullSchema
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
> at
> org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:59)
> at
> org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:130)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2785)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:118)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1120)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:147)
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NullPointerException: null of string in field c of
> nifi.fullSchema
> at
> org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:132)
> at
> org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:126)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
> at
> org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
> at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
> ... 17 common frames omitted
> Caused by: java.lang.NullPointerException: null
> {code}
> I worked around this by changing the type of the new fields from "string" to
> ["null", "string"] and then Avro seems to fill in the default value.
> We should be able to work around this and give the default value to Avro
> before it gets to the point of a NPE.
> Template attached.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
