[
https://issues.apache.org/jira/browse/ARROW-14196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426167#comment-17426167
]
Jim Pivarski commented on ARROW-14196:
--------------------------------------
If it's changing a default, I can just set the option explicitly, right?
If it's changing how columns are named in the file (currently
"fieldA.list.fieldB.list", etc.), then that would require adjustment on my
side, but it would be an adjustment that depends on how the file was written,
not the pyarrow version, right? If so, I'd have to support both naming
conventions because both would exist in the wild.
If there is a change that Awkward Array has to adjust to, then now may be a
good time to do it because we're going to be rewriting the "from_parquet"
function soon, as part of integrating with Dask
https://github.com/ContinuumIO/dask-awkward
adopting fsspec, and using pyarrow's Dataset API (to replace our manual
implementation). If there's something that we can set to get the new behavior,
we can turn that on now while developing the new version and be ready for the
change.
> [C++][Parquet] Default to compliant nested types in Parquet writer
> ------------------------------------------------------------------
>
> Key: ARROW-14196
> URL: https://issues.apache.org/jira/browse/ARROW-14196
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Parquet
> Reporter: Joris Van den Bossche
> Priority: Major
>
> In C++ there is already an option to get the "compliant_nested_types" (to
> have the list columns follow the Parquet specification), and ARROW-11497
> exposed this option in Python.
> This is still set to False by default, but in the source it says "TODO: At
> some point we should flip this.", and in ARROW-11497 there was also some
> discussion about what it would take to change the default.
> cc [~emkornfield] [~apitrou]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
