Shivraj Devidas Wabale created BEAM-14236:
---------------------------------------------
Summary: [Python] Write to Parquet support for list to conform
with Apache Parquet specification
Key: BEAM-14236
URL: https://issues.apache.org/jira/browse/BEAM-14236
Project: Beam
Issue Type: Bug
Components: io-py-parquet
Reporter: Shivraj Devidas Wabale
ARROW-11497 The pyarrow parquet writer now support the list type contains 3
level where the middle level, named {{{}list{}}}, must be a repeated group with
a single field named _{{element.
[https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#lists|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#lists,]}}_
I think we can simply populate it to [WriteToParquet
|https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/parquetio.py#L358]by
adding additional flag (use_compliant_nested_type) to
conform with Apache Parquet specification.
h4.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
