Joseph Percivall created NIFI-1253:
--------------------------------------
Summary: ConvertJSONToAvro doesn't fully validate schema before
scheduling
Key: NIFI-1253
URL: https://issues.apache.org/jira/browse/NIFI-1253
Project: Apache NiFi
Issue Type: Improvement
Reporter: Joseph Percivall
Priority: Minor
ConvertJSONToAvro takes in a JSON file and has an Avro schema as a parameter.
You can create a "valid" schema (as verified here[1]) but once the processor is
triggered it encounters an IllegalArgumentException, stack trace below. I used
this schema:
{noformat}
{
"name" :"results",
"type" : "array",
"items" :
{ "name" : "user",
"type" : "record",
"fields" : [
{"name": "gender", "type" : "string"},
{
"name" : "name",
"type" : {
"name": "name",
"type" : "record",
"fields" : [
{"name" : "title", "type" : "string"},
{"name" : "first", "type" : "string"},
{"name" : "last", "type" : "string"}
]
}
}
]
}
}
{noformat}
{noformat}
java.lang.IllegalArgumentException: Schemas for JSON files should be record
at
org.kitesdk.shaded.com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
~[na:na]
at
org.kitesdk.data.spi.filesystem.JSONFileReader.initialize(JSONFileReader.java:84)
~[na:na]
at
org.apache.nifi.processors.kite.ConvertJSONToAvro$1.process(ConvertJSONToAvro.java:144)
~[na:na]
at
org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2171)
~[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at
org.apache.nifi.processors.kite.ConvertJSONToAvro.onTrigger(ConvertJSONToAvro.java:139)
~[na:na]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146)
~[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139)
[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49)
[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119)
[nifi-framework-core-0.4.0-SNAPSHOT.jar:0.4.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_60]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[na:1.8.0_60]
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_60]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]
{noformat}
[1] https://json-schema-validator.herokuapp.com/avro.jsp
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
