hinxx opened a new issue, #34486:
URL: https://github.com/apache/arrow/issues/34486

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   I can not seem to convert pyarrow sparse_union table to pandas dataframe:
   
   ```
   >>> table
   pyarrow.Table
   name: string
   timestamp: timestamp[ns]
   value: sparse_union<_union_0: int64=0, _union_1: double=1, _union_2: 
string=2>
     child 0, _union_0: int64
     child 1, _union_1: double
     child 2, _union_2: string
   ----
   name: [["row1","row2","row3","row4","row5","row6"]]
   timestamp: [[2022-07-15 12:40:13.439549952,2022-07-15 
12:40:13.439546880,2023-02-08 09:13:32.287076352,2023-02-08 
09:13:32.587076352,2022-07-07 14:23:10.092787968,2022-07-15 12:40:13.839546624]]
   value: [  -- is_valid: all not null  -- type_ids: [1,1,0,0,2,1]
     -- child 0 type: int64
   [null,null,5,53,null,null]
     -- child 1 type: double
   [0.011021,-32580,null,null,null,-33580]
     -- child 2 type: string
   [null,null,null,null,"3.10.0",null]]
   
   
   >>> df=table.to_pandas()
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "pyarrow/array.pxi", line 835, in 
pyarrow.lib._PandasConvertible.to_pandas
       return self._to_pandas(options, categories=categories,
     File "pyarrow/table.pxi", line 4087, in pyarrow.lib.Table._to_pandas
       mgr = table_to_blockmanager(
     File "/arrow/python/pyarrow/pandas_compat.py", line 820, in 
table_to_blockmanager
       blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
     File "/arrow/python/pyarrow/pandas_compat.py", line 1169, in 
_table_to_blocks
       result = pa.lib.table_to_blocks(options, block_table, categories,
     File "pyarrow/table.pxi", line 2743, in pyarrow.lib.table_to_blocks
       check_status(
     File "pyarrow/error.pxi", line 121, in pyarrow.lib.check_status
       raise ArrowNotImplementedError(message)
   pyarrow.lib.ArrowNotImplementedError: No known equivalent Pandas block for 
Arrow data of type sparse_union<_union_0: int64=0, _union_1: double=1, 
_union_2: string=2> is known.
   
   ```
   
   Looks like it is not supported/implemented; is there a workaround/solution 
for this?
   
   ### Component(s)
   
   Integration, Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to