Ken Tore Tallakstad created NIFI-5755:
-----------------------------------------
Summary: Allow PutParquet prosessor to dynamically set config
properties for parquet lib
Key: NIFI-5755
URL: https://issues.apache.org/jira/browse/NIFI-5755
Project: Apache NiFi
Issue Type: New Feature
Components: Core Framework
Affects Versions: 1.7.1
Reporter: Ken Tore Tallakstad
PutParquet (NiFi 1.7.1), will fail to write parquet if incoming records contain
arrays with null elements:
Schema snippet example:
{
"name": "myfield",
"type" : ["null", \{ "type" : "array", "items" :
["null","string"], "default": null } ], "default": null
},
And a corresponding data example:
"myfield" : [ null, "value1" ],
"myfield" : [ null, "value2" ],
"myfield" : [ "value3", null, "value4" ],
Avro does not seem to have a problem with this, but Put parquet fails with the
following error: "Array contains a null element at X".
There is a parquet config to allow this:
parquet.avro.write-old-list-structure=false
If the processor would have a box for toggling this value, or a text box to
pass configs in general would be great! !parq2.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
