PCClimate commented on issue #37517:
URL: https://github.com/apache/arrow/issues/37517#issuecomment-1714913535
pyarrow version is 12.0.1.
This is the schema:
```
`id: string
updatetime: string
version: int32
names: map<string, list<array_element: map<string, string
('array_element')>> ('names')>
child 0, names: struct<key: string not null, value: list<array_element:
map<string, string ('array_element')>>> not null
child 0, key: string not null
child 1, value: list<array_element: map<string, string
('array_element')>>
child 0, array_element: map<string, string ('array_element')>
child 0, array_element: struct<key: string not null, value:
string> not null
child 0, key: string not null
child 1, value: string
categories: struct<main: string, alternate: list<array_element: string>>
child 0, main: string
child 1, alternate: list<array_element: string>
child 0, array_element: string
confidence: double
websites: list<array_element: string>
child 0, array_element: string
socials: list<array_element: string>
child 0, array_element: string
emails: list<array_element: string>
child 0, array_element: string
phones: list<array_element: string>
child 0, array_element: string
brand: struct<names: map<string, list<array_element: map<string, string
('array_element')>> ('names')>, wikidata: string>
child 0, names: map<string, list<array_element: map<string, string
('array_element')>> ('names')>
child 0, names: struct<key: string not null, value:
list<array_element: map<string, string ('array_element')>>> not null
child 0, key: string not null
child 1, value: list<array_element: map<string, string
('array_element')>>
child 0, array_element: map<string, string ('array_element')>
child 0, array_element: struct<key: string not null,
value: string> not null
child 0, key: string not null
child 1, value: string
child 1, wikidata: string
addresses: list<array_element: map<string, string ('array_element')>>
child 0, array_element: map<string, string ('array_element')>
child 0, array_element: struct<key: string not null, value: string>
not null
child 0, key: string not null
child 1, value: string
sources: list<array_element: map<string, string ('array_element')>>
child 0, array_element: map<string, string ('array_element')>
child 0, array_element: struct<key: string not null, value: string>
not null
child 0, key: string not null
child 1, value: string
bbox: struct<minx: double, maxx: double, miny: double, maxy: double>
child 0, minx: double
child 1, maxx: double
child 2, miny: double
child 3, maxy: double
geometry: binary
-- schema metadata --
writer.time.zone: 'UTC'`
```
`table.to_pandas()`
Returns:
```
---------------------------------------------------------------------------
ArrowNotImplementedError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_66740\1919897604.py in <module>
----> 1 table.to_pandas()
~\Anaconda3\lib\site-packages\pyarrow\array.pxi in
pyarrow.lib._PandasConvertible.to_pandas()
~\Anaconda3\lib\site-packages\pyarrow\table.pxi in
pyarrow.lib.Table._to_pandas()
~\Anaconda3\lib\site-packages\pyarrow\pandas_compat.py in
table_to_blockmanager(options, table, categories, ignore_metadata, types_mapper)
818 _check_data_column_metadata_consistency(all_columns)
819 columns = _deserialize_column_index(table, all_columns,
column_indexes)
--> 820 blocks = _table_to_blocks(options, table, categories,
ext_columns_dtypes)
821
822 axes = [columns, index]
~\Anaconda3\lib\site-packages\pyarrow\pandas_compat.py in
_table_to_blocks(options, block_table, categories, extension_columns)
1166 # Convert an arrow table to Block from the internal pandas API
1167 columns = block_table.column_names
-> 1168 result = pa.lib.table_to_blocks(options, block_table, categories,
1169 list(extension_columns.keys()))
1170 return [_reconstruct_block(item, columns, extension_columns)
~\Anaconda3\lib\site-packages\pyarrow\table.pxi in
pyarrow.lib.table_to_blocks()
~\Anaconda3\lib\site-packages\pyarrow\error.pxi in pyarrow.lib.check_status()
ArrowNotImplementedError: Not implemented type for Arrow list to pandas:
map<string, string ('array_element')>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
