Martin Hynar created NIFI-9890:
----------------------------------
Summary: Record conversion fails in JoltTransformREcord
Key: NIFI-9890
URL: https://issues.apache.org/jira/browse/NIFI-9890
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.15.3
Reporter: Martin Hynar
Hello, I am struggling with JOLT transformation of record flow file. The flow
is quite simple
# Consume records from Kafka - ConsumeKafkaRecord, where reader is Avro Reader
Confluent Reference and writer is Json Record Set - this step is OK and
flowfiles are correctly sent to success.
# Transform records - using JoltTransformRecord, with JsonTreeReader and Json
Record Set Writer - the transformation is very simple, new field is created
from existing fields. On writing result, bellow exception is thrown. Worth
mentioning, the problematic field is not used in the transformation at all.
The exception is this
{code:java}
2022-04-07 03:32:40,170 ERROR [Timer-Driven Process Thread-1]
o.a.n.p.jolt.record.JoltTransformRecord
JoltTransformRecord[id=2ea633ad-c1f3-1909-0000-0000686dd600] Unable to
transform
StandardFlowFileRecord[uuid=e32039eb-e7f9-46ea-a7a4-ad9cf3709b71,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1649063785609-18280,
container=default, section=872], offset=381094,
length=1143917],offset=0,name=e32039eb-e7f9-46ea-a7a4-ad9cf3709b71,size=1143917]
due to
org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
Cannot convert value [[Ljava.lang.Object;@d786ea4] of type
CHOICE[ARRAY[STRING], ARRAY[INT]] to Map for field IgnoreIssues because the
type is not supported:
org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
Cannot convert value [[Ljava.lang.Object;@d786ea4] of type
CHOICE[ARRAY[STRING], ARRAY[INT]] to Map for field IgnoreIssues because the
type is not supported
org.apache.nifi.serialization.record.util.IllegalTypeConversionException:
Cannot convert value [[Ljava.lang.Object;@d786ea4] of type
CHOICE[ARRAY[STRING], ARRAY[INT]] to Map for field IgnoreIssues because the
type is not supported
at
org.apache.nifi.serialization.record.util.DataTypeUtils.convertRecordFieldtoObject(DataTypeUtils.java:858)
at
org.apache.nifi.processors.jolt.record.JoltTransformRecord.transform(JoltTransformRecord.java:409)
at
org.apache.nifi.processors.jolt.record.JoltTransformRecord.onTrigger(JoltTransformRecord.java:334)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1273)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:103)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834) {code}
What I found about the field is that in the record, there are these values
{noformat}
"IgnoreIssues" : [ ],
"IgnoreIssues" : [ ],
"IgnoreIssues" : [ 0 ],
"IgnoreIssues" : [ 0 ],
"IgnoreIssues" : [ ],
"IgnoreIssues" : [ ],
{noformat}
so, single value arrays mixed with empty arrays.
Also, what I tried was to redirect all failed flow files into SplitRecord
processor with max records = 1. When these single record flowfiles were
returned to the same transformation, they passed.
The array is very often empty and I have flow files that pass the
transformation, I think it is when all records have empty array in this field.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
