[
https://issues.apache.org/jira/browse/ARROW-8273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney closed ARROW-8273.
-------------------------------
Resolution: Duplicate
Dup of ARROW-7365
> Fail to convert an integer list (arrow) to pandas
> -------------------------------------------------
>
> Key: ARROW-8273
> URL: https://issues.apache.org/jira/browse/ARROW-8273
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Jonathan mercier
> Priority: Minor
>
> Dear,
>
> I have an arrow table and one of columns as an arrow list(size=2)
> When I try to convert the table to pandas I got an ArrowNotImplementedError
> error
> below minimal case:
> {code:java}
> from pyarrow import Schema, Table, int64, list_, schema, array
> fields = [('foo', list_(int64(), 2),)]
> sc = schema(fields)
> foo_column = [[1,2], [3,4]]
> columns = [foo_column]
> a_table = Table.from_arrays(arrays=columns, schema=sc)
> df = a_table.to_pandas()
> ---------------------------------------------------------------------------
> ArrowNotImplementedError Traceback (most recent call last)
> <ipython-input-13-87e390bb8b12> in <module>
> ----> 1 df2 = a_table.to_pandas()
> ~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/array.pxi in
> pyarrow.lib._PandasConvertible.to_pandas()
> ~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/table.pxi in
> pyarrow.lib.Table._to_pandas()
> ~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/pandas_compat.py
> in table_to_blockmanager(options, table, categories, ignore_metadata,
> types_mapper)
> 764 _check_data_column_metadata_consistency(all_columns)
> 765 columns = _deserialize_column_index(table, all_columns, column_indexes)
> --> 766 blocks = _table_to_blocks(options, table, categories,
> ext_columns_dtypes)
> 767
> 768 axes = [columns, index]
> ~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/pandas_compat.py
> in _table_to_blocks(options, block_table, categories, extension_columns)
> 1099 columns = block_table.column_names
> 1100 result = pa.lib.table_to_blocks(options, block_table, categories,
> -> 1101 list(extension_columns.keys()))
> 1102 return [_reconstruct_block(item, columns, extension_columns)
> 1103 for item in result]
> ~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/table.pxi in
> pyarrow.lib.table_to_blocks()
> ~/somewhere/foo/venv/lib64/python3.7/site-packages/pyarrow/error.pxi in
> pyarrow.lib.check_status()
> ArrowNotImplementedError: No known equivalent Pandas block for Arrow data of
> type fixed_size_list<item: int64>[2] is known
> {code}
>
> If I try to convert python list (column) to array that segfault
> {code:java}
> from pyarrow import Schema, Table, int64, list_, schema, array
> fields = [('foo', list_(int64(), 2),)]
> sc = schema(fields)
> foo_column = [[1,2], [3,4]]
> columns = [foo_column]
> a_table = Table.from_arrays(arrays=columns, schema=sc)
> df = a_table.to_pandas(){code}
>
> If I remove the fixed size that works:
> {code:java}
> In [1]:
> from pyarrow import Schema, Table, int64, list_, schema, array
> fields = [('foo', list_(int64()),)]
> sc = schema(fields)
> foo_column = [[1,2], [3,4]]
> columns = [array(foo_column)]
> a_table = Table.from_arrays(arrays=columns, schema=sc)
> df = a_table.to_pandas()
> df
> Out [1]:
>
>
>
> foo
> 0 [1, 2]
> 1 [3, 4]{code}
>
> Thanks for your help
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
