[
https://issues.apache.org/jira/browse/PARQUET-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Antoine Pitrou updated PARQUET-1945:
------------------------------------
Component/s: (was: parquet-format)
parquet-mr
> Add an option to allow auto conversion from empty fields to NULL
> ----------------------------------------------------------------
>
> Key: PARQUET-1945
> URL: https://issues.apache.org/jira/browse/PARQUET-1945
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-mr
> Reporter: Zheng Shao
> Priority: Minor
>
> Right now, Parquet Writer throws out an exception:
> {{Parquet record is malformed: empty fields are illegal, the field should be
> ommited completely instead}}
> when an empty field (array or struct or map I guess?) is written.
> The suggestion here is to add an option "auto_convert_empty_fields_to_null"
> that convert empty fields to null automatically on write.
> The LOC to change is
> [here:|https://sourcegraph.com/github.com/apache/parquet-mr/-/blob/parquet-column/src/main/java/org/apache/parquet/io/MessageColumnIO.java#L328]
> {quote}{{if (emptyField) {}}
> {{ {{ throw new ParquetEncodingException("empty fields are illegal, the
> field should be ommited completely instead");}}}}
> {{}}}{quote}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
