[
https://issues.apache.org/jira/browse/ARROW-7731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17220602#comment-17220602
]
Artem KOZHEVNIKOV commented on ARROW-7731:
------------------------------------------
but the concat still does not work :
{code:python}
import pyarrow as pa
import pyarrow.parquet as pq
l1 = pa.array([list(range(100))] * 10**7, type=pa.list_(pa.int16()))
tt = pa.Table.from_pydict({'big': pa.chunked_array([l1] * 10)}) # if concat,
offset will overflow int32
tt.combine_chunks()
---------------------------------------------------------------------------
ArrowInvalid Traceback (most recent call last)
<ipython-input-3-39fcc668ae13> in <module>
----> 1 tt.combine_chunks()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/table.pxi in
pyarrow.lib.Table.combine_chunks()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/error.pxi in
pyarrow.lib.pyarrow_internal_check_status()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/error.pxi in
pyarrow.lib.check_status()
ArrowInvalid: offset overflow while concatenating arrays
{code}
> [C++][Parquet] Support LargeListArray
> -------------------------------------
>
> Key: ARROW-7731
> URL: https://issues.apache.org/jira/browse/ARROW-7731
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: marc abboud
> Assignee: Micah Kornfield
> Priority: Major
> Labels: parquet
> Fix For: 0.17.1
>
>
> For now it's not possible to write a pyarrow.Table containing a
> LargeListArray in parquet. The lines
> {code:java}
> from pyarrow import parquet
> import pyarrow as pa
> indices = [1, 2, 3]
> indptr = [0, 1, 2, 3]
> q = pa.lib.LargeListArray.from_arrays(indptr, indices)
> table = pa.Table.from_arrays([q], names=['no'])
> parquet.write_table(table, '/test'){code}
> yields the error
> {code:java}
> ArrowNotImplementedError: Unhandled type for Arrow to Parquet schema
> conversion: large_list<item: int64>
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
