Hi,
We have an issue with PutParquet (NiFi 1.7.1), well with the parquet lib to
be precise, and array type data containing null values.
This is a schema snippet of the field in question:
{
"name": "adresse",
"type" : ["null", { "type" : "array", "items" :
["null","string"], "default": null } ], "default": null
},
And a corresponding data example:
"adresse" : [ null, "value1" ],
"adresse" : [ null, "value2" ],
"adresse" : [ "value3", null, "value4" ],
Avro does not seem to have a problem with this and all our records pass,
but Put parquet fails with the following error: "Array contains a null
element at X".
Apparently there is a parquet config to allow
this: parquet.avro.write-old-list-structure=false. Any tips on how to set
it? Are there any other ways around this, besides stripping the raw data of
nulls in arrays?
Thanks!
KT :)
[image: parq1.png]
[image: parq2.png]
