I'm using Apache Nifi 1.9.2 and trying to post JSON content to a
BigQuery table.
There seems to be something wrong, sicne I get
2019-07-03 08:35:24,964 ERROR [Timer-Driven Process Thread-8]
o.a.n.p.gcp.bigquery.PutBigQueryBatch
PutBigQueryBatch[id=b2b1c6bf-016b-1000-e8c9-b3f9fb5b417e] null:
java.lang.NullPointerException
java.lang.NullPointerException: null
at
org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.mapToField(BigQueryUtils.java:42)
at
org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.listToFields(BigQueryUtils.java:68)
at
org.apache.nifi.processors.gcp.bigquery.BigQueryUtils.schemaFromString(BigQueryUtils.java:80)
at
org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:277)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1162)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:209)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Where can it come from ? And how can i fix it ?
From the stack, I'm understanding there is something wrong with my
BigQuery schema (which is however recognized as valid by BigQuery).
My schema is
[
{
"name": "Consent",
"type": "record",
"fields": [
{
"name": "id",
"type": "STRING"
},
{
"name": "identity",
"type": "record",
"fields": [
{
"name": "id",
"type": "STRING"
},
{
"name": "type",
"type": "STRING"
},
{
"name": "businessUnit",
"type": "STRING"
}
]
},
{
"name": "finality",
"type": "STRING"
},
{
"name": "source",
"type": "record",
"fields": [
{
"name": "id",
"type": "STRING"
},
{
"name": "type",
"type": "STRING"
},
{
"name": "origin",
"type": "STRING"
},
{
"name": "collaborator",
"type": "record",
"fields": [
{
"name": "id",
"type": "STRING"
},
{
"name": "type",
"type": "STRING"
}
]
}
]
},
{
"name": "consentDate",
"type": "TIMESTAMP"
},
{
"name": "expiryDate",
"type": "TIMESTAMP"
},
{
"name": "expired",
"type": "BOOLEAN"
},
{
"name": "createdBy",
"type": "STRING"
},
{
"name": "createdDate",
"type": "TIMESTAMP"
}
]
}
]
What can cause the trouble ?
Thanks
