[
https://issues.apache.org/jira/browse/NIFI-12883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17825353#comment-17825353
]
Daniel Stieglitz edited comment on NIFI-12883 at 3/11/24 3:18 PM:
------------------------------------------------------------------
[~exceptionfactory] Thanks for the reference. I wasn't aware of that. It is
interesting though as I was able to overcome this issue by using
[jackson-dataformats-binary|https://github.com/FasterXML/jackson-dataformats-binary/tree/master/avro#generating-avro-schema-from-pojo-definition]
to generate an Avro schema from NIFI's POJO
{code:java}
org.apache.nifi.web.api.entity.ControllerServiceEntity
{code}
which I then used in JsonTreeReader to read the attached file successfully.
This got me thinking if it is possible to generate Avro schemas from POJOs then
perhaps Apache NIFI should include in the distribution the generated Avro
schemas of all the objects returned by the Rest API in order to facilitate
reading the Rest API responses by JsonTreeReader and acting on them. Is that
something which should be included in the distribution or perhaps a different
artifact?
was (Author: JIRAUSER294662):
[~exceptionfactory] Thanks for the reference. I wasn't aware of that. It is
interesting though as I was able to overcome this issue by using
[jackson-dataformats-binary|https://github.com/FasterXML/jackson-dataformats-binary/tree/master/avro#generating-avro-schema-from-pojo-definition]
to generate an Avro schema from NIFI's POJO
{code:java}
org.apache.nifi.web.api.entity.ControllerServiceEntity
{code}
which I then used in JsonTreeReader to read the attached file successfully.
This got me thinking if it is possible to generate Avro schemas from POJOs then
perhaps Apache NIFI should include in the distribution the generated Avro
schemas of all the objects returned by the Rest API in order to facilitate
reading the Rest API responses by JsonTreeReader and acting on them.
> JsonTreeReader not able to infer schema from JSON of a ControllerServiceEntity
> ------------------------------------------------------------------------------
>
> Key: NIFI-12883
> URL: https://issues.apache.org/jira/browse/NIFI-12883
> Project: Apache NiFi
> Issue Type: Bug
> Reporter: Daniel Stieglitz
> Priority: Major
> Attachments: ControllerServiceEntity.json
>
>
> When trying to use the ConvertRecord processor configured with a
> JsonTreeReader whose Schema
> Access Strategy is to Infer schema and configured with a JsonRecordSetWriter
> to write the
> schema out all in order to read and write a ControllerServiceEntity JSON
> object returned by the NIFI Rest API and produce the Avro schema, I got the
> following stacktrace
> below. I have attached the ControllerServiceEntity JSON which I tried to
> convert.
> {code:java}
> 2024-03-08 17:41:26,198 ERROR [Timer-Driven Process Thread-2]
> o.a.n.processors.standard.ConvertRecord
> ConvertRecord[id=1f213eb6-018e-1000-e76a-b9ac6041ed48] Failed to process
> StandardFlowFileRecord[uuid=45aa31af-0850-42be-9f9e-05001acbf8f2,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1709914518257-1, container=default,
> section=1], offset=34287,
> length=14358],offset=0,name=sampleAfterDisablingStandardJsonSchemaRegistry.json,size=14358];
> will route to failure
> org.apache.avro.SchemaParseException: Illegal character in:
> component_descriptors_JSON Schema Version_allowableValues_allowableValueType
> at org.apache.avro.Schema.validateName(Schema.java:1625)
> at org.apache.avro.Schema.access$400(Schema.java:94)
> at org.apache.avro.Schema$Name.<init>(Schema.java:713)
> at org.apache.avro.Schema.createRecord(Schema.java:226)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:287)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:211)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:284)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroField(AvroTypeUtil.java:130)
> at org.apache.nifi.avro.AvroTypeUtil.buildAvroSchema(AvroTypeUtil.java:122)
> at
> org.apache.nifi.avro.AvroTypeUtil.extractAvroSchema(AvroTypeUtil.java:102)
> at
> org.apache.nifi.schema.access.WriteAvroSchemaAttributeStrategy.lambda$new$0(WriteAvroSchemaAttributeStrategy.java:36)
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.lambda$newMappingFunction$2(LocalLoadingCache.java:145)
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.lambda$doComputeIfAbsent$14(BoundedLocalCache.java:2406)
> at
> java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1916)
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.doComputeIfAbsent(BoundedLocalCache.java:2404)
> at
> com.github.benmanes.caffeine.cache.BoundedLocalCache.computeIfAbsent(BoundedLocalCache.java:2387)
> at
> com.github.benmanes.caffeine.cache.LocalCache.computeIfAbsent(LocalCache.java:108)
> at
> com.github.benmanes.caffeine.cache.LocalLoadingCache.get(LocalLoadingCache.java:56)
> at
> org.apache.nifi.schema.access.WriteAvroSchemaAttributeStrategy.getAttributes(WriteAvroSchemaAttributeStrategy.java:53)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:151)
> at
> org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:254)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.access$100(StandardControllerServiceInvocationHandler.java:38)
> at
> org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler$ProxiedReturnObjectInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:240)
> at jdk.proxy18/jdk.proxy18.$Proxy180.write(Unknown Source)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:153)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:3432)
> at
> org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1361)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:247)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:102)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
