[
https://issues.apache.org/jira/browse/NIFI-5664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16647756#comment-16647756
]
ASF GitHub Bot commented on NIFI-5664:
--------------------------------------
Github user MikeThomsen commented on the issue:
https://github.com/apache/nifi/pull/3049
@snagacarl since this was for the Mongo lookup service, wanted to make sure
that you're aware that in 1.8 there will be a slight breaking change. The
"configuration" controller service and the lookup service are now cleanly
separated so that the lookup service can be cheerfully ignorant of connection
details like the URL, SSL and authentication details. That way plugging in a
new cluster or something will be faster and easier for Mongo users.
> LookupRecord cannot handle JSON array properties from MongoDBLookupService
> --------------------------------------------------------------------------
>
> Key: NIFI-5664
> URL: https://issues.apache.org/jira/browse/NIFI-5664
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
> Affects Versions: 1.8.0, 1.7.1
> Reporter: Carl Gieringer
> Priority: Major
> Fix For: 1.8.0
>
>
> We are using a LookupRecord processor to add an array property to a Record.
> The LookupRecord's `Lookup Service` is a MongoDBLookupService. The idea is
> for the LookupRecord to query Mongo documents by the Record's `_id`, to
> extract the array property at `f` on that document, and to insert that array
> into the Record at the RecordPath `f`. This flow results in the conversion
> error copied below, which completely prevents the FlowFile from proceeding
> (routes to Failure/Unmatched.) It appears that the `MongoDBLookupService` (or
> perhaps its `MongoDBControllerService`) stores Mongo document arrays as
> `ArrayList`s, which `DataTypeUtils#toArray` does not support. A simple fix
> appears to be to add a few lines to `DataTypeUtils#toArray` to support
> `ArrayList`s. I will provide a PR for this issue.
> This issue appears in 1.7.1, and is extant in 1.8.0-SNAPSHOT (2018-10-02
> b4c8e0179). I am not aware of any workaround to enrich a Record with an array
> from Mongo.
> LookupRecord Configuration
> ||Property||Value||
> |Result RecordPath|/f|
> |Record Result Contents|Insert Entire Record|
> |_id|/_id|
> MongoDBLookupService Configuration
> ||Property||Value||
> |Lookup Value Field|f|
> Conversion error (with irrelevant Mongo document and schema properties
> elided):
> {code:java}
> 2018-10-04 15:00:31,348 ERROR [Timer-Driven Process Thread-1]
> o.a.n.processors.standard.LookupRecord
> LookupRecord[id=339bf72c-91c3-36f0-ac14-bc8b734e2582] Failed to write
> MapRecord[{... f=[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}],
> i=[Ljava.lang.Object;@35dee8fc, ...] with schema {
> "namespace": "nifi",
> "name": "MongoPosting",
> "type": "record",
> "fields": [
> ...
> { "name": "f", "type": [
> "null",
> { "type": "array", "items": {
> "type": "record", "name": "PostingFeatureDetails", "fields": [
> { "name": "fid", "type": "string" },
> { "name": "l" , "type": ["null", "double"] }
> ]}
> }
> ]},
> ...
> ]
> } as a JSON Object due to
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f:
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.toArray(DataTypeUtils.java:347)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:115)
> at
> org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:284)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:187)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:136)
> at
> org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
> at
> org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:148)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2218)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2186)
> at
> org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:121)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> 2018-10-04 15:00:31,349 ERROR [Timer-Driven Process Thread-1]
> o.a.n.processors.standard.LookupRecord
> LookupRecord[id=339bf72c-91c3-36f0-ac14-bc8b734e2582] Failed to process
> StandardFlowFileRecord[uuid=29b498fe-787f-4c62-a237-f95a8ebcb0b2,claim=StandardContentClaim
> [resourceClaim=StandardResourceClaim[id=1538665215854-2, container=default,
> section=2], offset=100486,
> length=635840],offset=624238,name=1675395032391,size=11602]:
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f
> org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
> Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}},
> Document{{fid=5762adf32fc74d0418104201, l=2.12}},
> Document{{fid=5762addd18b8b607802fec6f, l=3.32}},
> Document{{fid=587542dd638e870838a79d08, l=1.0}},
> Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class
> java.util.ArrayList to Object Array for field f
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.toArray(DataTypeUtils.java:347)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
> at
> org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:115)
> at
> org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:284)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:187)
> at
> org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:136)
> at
> org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
> at
> org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:148)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2218)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2186)
> at
> org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:121)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
