anemohan opened a new issue, #39172:
URL: https://github.com/apache/arrow/issues/39172
### Describe the usage question you have. Please include as many useful
details as possible.
Hello, I use a schema file defining all the column types. I have a column
with a list of dicts and when I try to call the dataset with this schema I get
an error stating:
```
The original error is below:
------------------------
ArrowNotImplementedError('Unsupported cast from list<item: struct<count:
int64, gender: string, name: string, probability: double>> to utf8 using
function cast_string')
Traceback:
---------
File "/opt/conda/lib/python3.10/site-packages/dask/dataframe/utils.py",
line 193, in raise_on_meta_error
yield
File "/opt/conda/lib/python3.10/site-packages/dask/dataframe/core.py",
line 6897, in _emulate
return func(*_extract_meta(args, True), **_extract_meta(kwargs, True))
File
"/opt/conda/lib/python3.10/site-packages/intakewrapper/MHParquetSource.py",
line 67, in __call__
df = pd.read_parquet(
File "/opt/conda/lib/python3.10/site-packages/pandas/io/parquet.py", line
509, in read_parquet
return impl.read(
File "/opt/conda/lib/python3.10/site-packages/pandas/io/parquet.py", line
227, in read
pa_table = self.api.parquet.read_table(
File
"/opt/conda/lib/python3.10/site-packages/pyarrow/parquet/__init__.py", line
2780, in read_table
return dataset.read(columns=columns, use_threads=use_threads,
File
"/opt/conda/lib/python3.10/site-packages/pyarrow/parquet/__init__.py", line
2443, in read
table = self._dataset.to_table(
File "pyarrow/_dataset.pyx", line 304, in pyarrow._dataset.Dataset.to_table
File "pyarrow/_dataset.pyx", line 2549, in
pyarrow._dataset.Scanner.to_table
File "pyarrow/error.pxi", line 144, in
pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 121, in pyarrow.lib.check_status
```
Can someone help me with this?
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
