Mark Payne created NIFI-11621:
---------------------------------
Summary: Inferring schema for JSON fails when there's a CHOICE of
different ARRAY types
Key: NIFI-11621
URL: https://issues.apache.org/jira/browse/NIFI-11621
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Mark Payne
Assignee: Mark Payne
Fix For: 1.latest, 2.latest
>From Apache Slack:
>https://apachenifi.slack.com/archives/C0L9VCD47/p1685553667778359?thread_ts=1685461745.470939&cid=C0L9VCD47
When using ConvertRecord with a JSON Reader and an Avro Writer, when inferring
the JSON schema, each of the following two records works properly:
{code}
{"test_record":{"array_test_record":{"test_array":[]}}}
{code}
{code}
{"test_record":{"array_test_record":{"test_array":["test"]}}}
{code}
However, when combined into a single FlowFile:
{code}
{"test_record":{"array_test_record":{"test_array":[]}}}
{"test_record":{"array_test_record":{"test_array":["test"]}}}
{code}
It fails with a NullPointerException:
{code}
2023-05-31 13:51:35,632 ERROR [Timer-Driven Process Thread-8]
o.a.n.processors.standard.ConvertRecord
ConvertRecord[id=72e564dc-0188-1000-360a-9f86b50ec8ac] Failed to process
StandardFlowFileRecord[uuid=9bf4f0fb-0942-48ba-8a16-0dbd98db3f97,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1685554864966-1, container=default,
section=1], offset=3278,
length=117],offset=0,name=9bf4f0fb-0942-48ba-8a16-0dbd98db3f97,size=117]; will
route to failure
org.apache.nifi.processor.exception.ProcessException: Could not determine the
Avro Schema to use for writing the content
at
org.apache.nifi.avro.AvroRecordSetWriter.createWriter(AvroRecordSetWriter.java:154)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:105)
at com.sun.proxy.$Proxy177.createWriter(Unknown Source)
at
org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:150)
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3441)
at
org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1360)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:243)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.apache.nifi.schema.access.SchemaNotFoundException: Failed to
compile Avro Schema
at
org.apache.nifi.avro.AvroRecordSetWriter.createWriter(AvroRecordSetWriter.java:145)
... 21 common frames omitted
Caused by: java.lang.NullPointerException: null
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:208)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:241)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
at
org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:122)
at
org.apache.nifi.avro.AvroTypeUtil.extractAvroSchema(AvroTypeUtil.java:102)
at
org.apache.nifi.avro.AvroRecordSetWriter.createWriter(AvroRecordSetWriter.java:142)
... 21 common frames omitted
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
