[
https://issues.apache.org/jira/browse/NIFI-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17635936#comment-17635936
]
Mark Ward commented on NIFI-10660:
----------------------------------
Hi,
I have also faced this issue after updating from 1.15.3 to 1.18.0.
In my case incoming data was CSV, but this data was read correctly and
converted to Avro with an embedded schema. The problem arose when subsequent
processors read the Avro content using the embedded schema. Furthermore, the
alias had to contain a space.
See here for more info:
https://apachenifi.slack.com/archives/C0L9VCD47/p1668772654762409
Thanks
Mark
> AvroReader issue after upgrading to Nifi 1.18.0
> -----------------------------------------------
>
> Key: NIFI-10660
> URL: https://issues.apache.org/jira/browse/NIFI-10660
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.18.0
> Reporter: Sandro Berger
> Priority: Major
>
> After upgrading from 1.15.3 to 1.18.0 we saw some issue regarding AvroReader
> when the following conditions are met:
> # The AVRO schema in the flowFile uses the "aliases" attribute
> # There is a record in the flowFile where a field with set alias has a value
> of "null"
> The use of aliases and null values used to work for us before upgrading Nifi
> without issues
> Example schema:
> {code:java}
> {
> "type": "record",
> "name": "test",
> "fields": [
> { "name" : "id", "type" : "int", "aliases" : ["F1"] },
> { "name" : "field1", "type" : [ "string", "null" ], "aliases" : ["F2"]
> },
> { "name" : "field2", "type" : [ "string", "null" ], "aliases" : ["F3"] }
> ]
> }
> {code}
> Example data:
> {code:java}
> [ {
> "id" : 1,
> "field1" : "field1content",
> "field2" : "field2content"
> }, {
> "id" : 2,
> "field1" : null,
> "field2" : "field2content"
> } ]{code}
> Note: If we remove the "aliases" from the schema or make sure that there is
> never a field with a value of null, we don't have any errors in Nifi 1.18.0.
>
> ERROR log:
> {code:java}
> 2022-10-14 16:30:50,617 ERROR [Timer-Driven Process Thread-6]
> o.a.n.processors.standard.ConvertRecord
> ConvertRecord[id=c7ee1f0f-10ac-116e-9e4c-300530853ab8] Failed to process
> StandardFlow
> FileRecord[uuid=1d201fdb-e507-4d51-87c2-fbc895a450f3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1665753427357-4670,
> container=default, section=574], offset=18384,
> length=324],offset=0,name=1d201fdb-e507-4d51-87c2-fbc895a450f3,size=324];
> will route to failure
> org.apache.nifi.processor.exception.ProcessException: Could not parse
> incoming data
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:171)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3434)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1354)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: org.apache.nifi.serialization.MalformedRecordException: Error
> while getting next record
> at
> org.apache.nifi.avro.AvroRecordReader.nextRecord(AvroRecordReader.java:50)
> at
> org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50)
> at jdk.internal.reflect.GeneratedMethodAccessor150.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240)
> at com.sun.proxy.$Proxy277.nextRecord(Unknown Source)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:157)
> ... 13 common frames omitted
> Caused by: org.apache.avro.AvroRuntimeException: Not a valid schema field: F2
> at
> org.apache.avro.generic.GenericData$Record.get(GenericData.java:268)
> at
> org.apache.nifi.avro.AvroTypeUtil.convertAvroRecordToMap(AvroTypeUtil.java:916)
> at
> org.apache.nifi.avro.AvroTypeUtil.convertAvroRecordToMap(AvroTypeUtil.java:905)
> at
> org.apache.nifi.avro.AvroRecordReader.nextRecord(AvroRecordReader.java:43)
> ... 22 common frames omitted {code}
> The error mentions the alias of the field containing the null value.
> Could be related to the upgrade of avro introduced with Nifi 1.18.0:
> https://issues.apache.org/jira/browse/NIFI-10311
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
