Zheng Shao created PARQUET-1945:
-----------------------------------
Summary: Add an option to allow auto conversion from empty fields
to NULL
Key: PARQUET-1945
URL: https://issues.apache.org/jira/browse/PARQUET-1945
Project: Parquet
Issue Type: Improvement
Components: parquet-format
Reporter: Zheng Shao
Right now, Parquet Writer throws out an exception:
{{Parquet record is malformed: empty fields are illegal, the field should be
ommited completely instead}}
when an empty field (array or struct or map I guess?) is written.
The suggestion here is to add an option "auto_convert_empty_fields_to_null"
that convert empty fields to null automatically on write.
The LOC to change is
[here:|https://sourcegraph.com/github.com/apache/parquet-mr/-/blob/parquet-column/src/main/java/org/apache/parquet/io/MessageColumnIO.java#L328]
{{ if (emptyField) {}}
{{ throw new ParquetEncodingException("empty fields are illegal, the field
should be ommited completely instead");}}
{{ }}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
