DorotaDR opened a new issue, #34239: URL: https://github.com/apache/arrow/issues/34239
### Describe the bug, including details regarding any error messages, version, and platform. Using `pyArrow` `11.0.0`. Python 3.10 I used ArrowFlight to download the csv data: [titanic_pyarrow_issue.csv](https://github.com/apache/arrow/files/10768333/titanic_pyarrow_issue.csv). I saved the batches to pyarrow.Table and then tried to convert the Table to pandas: ``` info = readClient.get_flight_info( flight.FlightDescriptor.for_command(source_command)) for endpoint in info.endpoints: reader = readClient.do_get(endpoint.ticket) df, metadata = reader.read_chunk() data_table = pa.Table.from_batches(batches=[df] data = data_table.to_pandas() # the line fails # df = reader.read_pandas() # it also fails ``` The conversion to pandas type has failed with the following error: ``` File "/Users/[...]/python/flightdemo.py", line 71, in <module> data = data_table.to_pandas() File "pyarrow/array.pxi", line 830, in pyarrow.lib._PandasConvertible.to_pandas File "pyarrow/table.pxi", line 3990, in pyarrow.lib.Table._to_pandas File "/Users/[...]/lib/python3.9/site-packages/pyarrow/pandas_compat.py", line 820, in table_to_blockmanager blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes) File "/Users/[...]/lib/python3.9/site-packages/pyarrow/pandas_compat.py", line 1169, in _table_to_blocks result = pa.lib.table_to_blocks(options, block_table, categories, File "pyarrow/table.pxi", line 2646, in pyarrow.lib.table_to_blocks File "pyarrow/error.pxi", line 138, in pyarrow.lib.check_status pyarrow.lib.ArrowException: Unknown error: Wrapping Q� failed ``` Here is pickled `data_table` object: [data_unexpected_file.pickle.zip](https://github.com/apache/arrow/files/10768406/data_unexpected_file.pickle.zip) _To read the pickle file:_ _1. unzip_ _2. use pickle.load:_ ``` with open("data_unexpected_file.pickle", 'rb') as f: loaded_table = pickle.load(f) ``` It's worth to mention that `pandas` can read the csv file without any errors. ### Component(s) Python -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
