[
https://issues.apache.org/jira/browse/NIFI-10660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17636018#comment-17636018
]
Mark Payne commented on NIFI-10660:
-----------------------------------
Agreed [~joewitt] . Looks like Avro made some changes. Previously, we did
something like:
{code:java}
Object value = record.get(field.getFieldName());
if (value == null) {
for (String alias : field.getAliases()) {
value = record.get(alias);
if (value != null) {
break;
}
}
} {code}
This would give us the value from the Record, regardless of whether it came
from the field 'name' or an alias.
But now, if we call {{Record.get(alias)}} and the schema doesn't have that
alias, instead of returning {{null}} it throws an Exception.
I think we can work around this by first checking something like:
{code:java}
Field field = record.getSchema().getField(alias);
if (field != null) {
return record.get(alias);
}{code}
Will need to do some more testing though to confirm that this works as I think
it will.
> AvroReader issue after upgrading to Nifi 1.18.0
> -----------------------------------------------
>
> Key: NIFI-10660
> URL: https://issues.apache.org/jira/browse/NIFI-10660
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.18.0
> Reporter: Sandro Berger
> Priority: Critical
> Fix For: 1.19.0
>
>
> After upgrading from 1.15.3 to 1.18.0 we saw some issue regarding AvroReader
> when the following conditions are met:
> # The AVRO schema in the flowFile uses the "aliases" attribute
> # There is a record in the flowFile where a field with set alias has a value
> of "null"
> The use of aliases and null values used to work for us before upgrading Nifi
> without issues
> Example schema:
> {code:java}
> {
> "type": "record",
> "name": "test",
> "fields": [
> { "name" : "id", "type" : "int", "aliases" : ["F1"] },
> { "name" : "field1", "type" : [ "string", "null" ], "aliases" : ["F2"]
> },
> { "name" : "field2", "type" : [ "string", "null" ], "aliases" : ["F3"] }
> ]
> }
> {code}
> Example data:
> {code:java}
> [ {
> "id" : 1,
> "field1" : "field1content",
> "field2" : "field2content"
> }, {
> "id" : 2,
> "field1" : null,
> "field2" : "field2content"
> } ]{code}
> Note: If we remove the "aliases" from the schema or make sure that there is
> never a field with a value of null, we don't have any errors in Nifi 1.18.0.
>
> ERROR log:
> {code:java}
> 2022-10-14 16:30:50,617 ERROR [Timer-Driven Process Thread-6]
> o.a.n.processors.standard.ConvertRecord
> ConvertRecord[id=c7ee1f0f-10ac-116e-9e4c-300530853ab8] Failed to process
> StandardFlow
> FileRecord[uuid=1d201fdb-e507-4d51-87c2-fbc895a450f3,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1665753427357-4670,
> container=default, section=574], offset=18384,
> length=324],offset=0,name=1d201fdb-e507-4d51-87c2-fbc895a450f3,size=324];
> will route to failure
> org.apache.nifi.processor.exception.ProcessException: Could not parse
> incoming data
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:171)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3434)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1354)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
> at
> org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: org.apache.nifi.serialization.MalformedRecordException: Error
> while getting next record
> at
> org.apache.nifi.avro.AvroRecordReader.nextRecord(AvroRecordReader.java:50)
> at
> org.apache.nifi.serialization.RecordReader.nextRecord(RecordReader.java:50)
> at jdk.internal.reflect.GeneratedMethodAccessor150.invoke(Unknown
> Source)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240)
> at com.sun.proxy.$Proxy277.nextRecord(Unknown Source)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:157)
> ... 13 common frames omitted
> Caused by: org.apache.avro.AvroRuntimeException: Not a valid schema field: F2
> at
> org.apache.avro.generic.GenericData$Record.get(GenericData.java:268)
> at
> org.apache.nifi.avro.AvroTypeUtil.convertAvroRecordToMap(AvroTypeUtil.java:916)
> at
> org.apache.nifi.avro.AvroTypeUtil.convertAvroRecordToMap(AvroTypeUtil.java:905)
> at
> org.apache.nifi.avro.AvroRecordReader.nextRecord(AvroRecordReader.java:43)
> ... 22 common frames omitted {code}
> The error mentions the alias of the field containing the null value.
> Could be related to the upgrade of avro introduced with Nifi 1.18.0:
> https://issues.apache.org/jira/browse/NIFI-10311
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
