Gobi2511 commented on issue #34239:
URL: https://github.com/apache/arrow/issues/34239#issuecomment-2630835633
I'm also getting the same error but I'm getting this error when reading the
data from databricks which has some utf-8 encoded data. Sharring my exception
details below:
**Code**
```
sql_query = f""" SELECT Column FROM table LIMIT 10 """
host = os.getenv("DATABRICKS_HOST")
http_path = os.getenv("DATABRICKS_HTTP_PATH")
connection = sql.connect(server_hostname=host, http_path=http_path)
with conn.cursor() as cursor:
cursor.execute(sql_query)
response = cursor.fetchmany(1000)
```
**Error**
```
^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/my_agent/app_helper/db_helper.py",
line 35, in fetchmany
query_results = cursor.fetchmany(fetch_size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/my_agent_connectors/dq_databricks/dependencies/databricks/sql/client.py",
line 976, in fetchmany
return self.active_result_set.fetchmany(size)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/my_agent_connectors/dq_databricks/dependencies/databricks/sql/client.py",
line 1235, in fetchmany
return self._convert_arrow_table(self.fetchmany_arrow(size))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/my_agent_connectors/dq_databricks/dependencies/databricks/sql/client.py",
line 1161, in _convert_arrow_table
df = table_renamed.to_pandas(
^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/array.pxi", line 884, in
pyarrow.lib._PandasConvertible.to_pandas
File "pyarrow/table.pxi", line 4192, in pyarrow.lib.Table._to_pandas
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/pyarrow/pandas_compat.py",
line 776, in table_to_dataframe
blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/pyarrow/pandas_compat.py",
line 1131, in _table_to_blocks
return [_reconstruct_block(item, columns, extension_columns)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/pyarrow/pandas_compat.py",
line 1131, in <listcomp>
return [_reconstruct_block(item, columns, extension_columns)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/pyarrow/pandas_compat.py",
line 736, in _reconstruct_block
pd_ext_arr = pandas_dtype.__from_arrow__(arr)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/usr/local/airflow/.local/lib/python3.11/site-packages/pandas/core/arrays/string_.py",
line 230, in __from_arrow__
arr = arr.to_numpy(zero_copy_only=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "pyarrow/array.pxi", line 1581, in pyarrow.lib.Array.to_numpy
File "pyarrow/error.pxi", line 91, in pyarrow.lib.check_status
pyarrow.lib.ArrowException: Unknown error: Wrapping Ruta 204 � Zapote failed
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
