hinxx opened a new issue, #34486:
URL: https://github.com/apache/arrow/issues/34486
### Describe the bug, including details regarding any error messages,
version, and platform.
I can not seem to convert pyarrow sparse_union table to pandas dataframe:
```
>>> table
pyarrow.Table
name: string
timestamp: timestamp[ns]
value: sparse_union<_union_0: int64=0, _union_1: double=1, _union_2:
string=2>
child 0, _union_0: int64
child 1, _union_1: double
child 2, _union_2: string
----
name: [["row1","row2","row3","row4","row5","row6"]]
timestamp: [[2022-07-15 12:40:13.439549952,2022-07-15
12:40:13.439546880,2023-02-08 09:13:32.287076352,2023-02-08
09:13:32.587076352,2022-07-07 14:23:10.092787968,2022-07-15 12:40:13.839546624]]
value: [ -- is_valid: all not null -- type_ids: [1,1,0,0,2,1]
-- child 0 type: int64
[null,null,5,53,null,null]
-- child 1 type: double
[0.011021,-32580,null,null,null,-33580]
-- child 2 type: string
[null,null,null,null,"3.10.0",null]]
>>> df=table.to_pandas()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pyarrow/array.pxi", line 835, in
pyarrow.lib._PandasConvertible.to_pandas
return self._to_pandas(options, categories=categories,
File "pyarrow/table.pxi", line 4087, in pyarrow.lib.Table._to_pandas
mgr = table_to_blockmanager(
File "/arrow/python/pyarrow/pandas_compat.py", line 820, in
table_to_blockmanager
blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
File "/arrow/python/pyarrow/pandas_compat.py", line 1169, in
_table_to_blocks
result = pa.lib.table_to_blocks(options, block_table, categories,
File "pyarrow/table.pxi", line 2743, in pyarrow.lib.table_to_blocks
check_status(
File "pyarrow/error.pxi", line 121, in pyarrow.lib.check_status
raise ArrowNotImplementedError(message)
pyarrow.lib.ArrowNotImplementedError: No known equivalent Pandas block for
Arrow data of type sparse_union<_union_0: int64=0, _union_1: double=1,
_union_2: string=2> is known.
```
Looks like it is not supported/implemented; is there a workaround/solution
for this?
### Component(s)
Integration, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]