Rostislav Nedelchev created NIFI-8026:
-----------------------------------------
Summary: PutMongoRecord cannot handle deeply nested JSON documents.
Key: NIFI-8026
URL: https://issues.apache.org/jira/browse/NIFI-8026
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 1.12.1
Reporter: Rostislav Nedelchev
Attachments: nested_json.json
I am running Apache Nifi 1.12.1 to import data from S3 to MongoDB.
Unfortunately, the PutMongoRecord with JsonTreeReader cannot handle nested
JSONs. I have used the GenerateFlowFile Processor to isolate one such example
( [^nested_json.json] ) .
{code:java}
// org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
Cannot convert value
[MapRecord[{geoKoordinate=MapRecord[{breitengrad=MapRecord[{rawValue=53.523275}],
laengengrad=MapRecord[{rawValue=9.914175}]}],
gueltigkeit=MapRecord[{von=2018-12-03T00:00Z}],
planungsknoten=MapRecord[{geoKoordinate=MapRecord[{breitengrad=MapRecord[{rawValue=53.523275}],
laengengrad=MapRecord[{rawValue=9.914175}]}],
ortsangabeReferenz=MapRecord[{objektSchluessel=MapRecord[{sequence=1,
alpha=HAMBURG-WALTERSH DRA}],
techKey=MapRecord[{rawValue=800000000008925481}]}],
gueltigkeit=MapRecord[{von=2018-06-27T00:00Z}],
objektSchluessel=MapRecord[{sequence=1, alpha=HAMBURG-WALTERSHOF}],
techKey=MapRecord[{rawValue=800000000007210910}],
produktionsCode=MapRecord[{bahnverwaltungsnummer=80,
netzbahnstellennummer=010819}], bsReferenzen=[Ljava.lang.Object;@bdd9995,
name=MapRecord[{rawValue=Hamburg-Waltershof}],
rlCode=MapRecord[{rawValue=AWHOD}],
version=MapRecord[{gespeicherteInterneVersion=-1, externeVersion=1530090460,
interneVersion=0, gespeicherteExterneVersion=-1}]}],
mitbedienteGwps=[Ljava.lang.Object;@1b29f43b,
objektSchluessel=MapRecord[{sequence=2, alpha=AWHO CTB ohne Zoll}],
techKey=MapRecord[{rawValue=800000000007703732}],
produktionsCode=MapRecord[{bahnverwaltungsnummer=80, gwpNummer=1010,
netzbahnstellennummer=010819}], name=MapRecord[{rawValue=AWHO CTB Container}],
rlCode=MapRecord[{rawValue=AWHOD}],
version=MapRecord[{gespeicherteInterneVersion=-1, externeVersion=1543829665,
interneVersion=0, gespeicherteExterneVersion=-1}]}]] of type CHOICE[RECORD,
RECORD] to Map for field meldungsOrt because the type is not supported
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordFieldtoObject(DataTypeUtils.java:821)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordArrayToJavaArray(DataTypeUtils.java:860)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordFieldtoObject(DataTypeUtils.java:819)
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordFieldtoObject(DataTypeUtils.java:814)
at
org.apache.nifi.processors.mongodb.PutMongoRecord.onTrigger(PutMongoRecord.java:133)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
