sjperkins commented on pull request #10565: URL: https://github.com/apache/arrow/pull/10565#issuecomment-885786527
The Complex Extension Type now seems to be somewhat understood by the Python layer, but seems to be failing because there's no `__arrow_ext_class__` attribute, which seems to be [required ](https://arrow.apache.org/docs/python/extending_types.html )by the Python layer https://github.com/sjperkins/arrow/runs/3144270259?check_suite_focus=true#step:8:9416 ```traceback Traceback (most recent call last): File "/opt/conda/envs/arrow/lib/python3.7/multiprocessing/pool.py", line 121, in worker result = (True, func(*args, **kwds)) File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py", line 2533, in _threaded_conversion _check_pandas_roundtrip(df, use_threads=True) File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py", line 91, in _check_pandas_roundtrip nthreads=2 if use_threads else 1) File "pyarrow/table.pxi", line 1561, in pyarrow.lib.Table.from_pandas File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 607, in dataframe_to_arrays arrays[i] = maybe_fut.result() File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/_base.py", line 428, in result return self.__get_result() File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result raise self._exception File "/opt/conda/envs/arrow/lib/python3.7/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py", line 575, in convert_column result = pa.array(col, type=type_, from_pandas=True, safe=safe) File "pyarrow/array.pxi", line 302, in pyarrow.lib.array File "pyarrow/array.pxi", line 89, in pyarrow.lib._ndarray_to_array File "pyarrow/types.pxi", line 200, in pyarrow.lib.DataType.to_pandas_dtype File "pyarrow/array.pxi", line 2402, in pyarrow.lib.get_array_class_from_type AttributeError: 'pyarrow.lib.BaseExtensionType' object has no attribute '__arrow_ext_class__' ``` I **think** this may be happening because of the following lines https://github.com/apache/arrow/blob/42e5726b543135f18db1560278e062b9a7941f5c/python/pyarrow/public-api.pxi#L110-L116 which are invoked in this location: https://github.com/apache/arrow/blob/42e5726b543135f18db1560278e062b9a7941f5c/python/pyarrow/array.pxi#L2400-L2402 So at this stage, it doesn't look like the underlying C++ Extension Type is fully exposed in the Python layer. For example, the `__arrow_ext_class__` should return some Python wrapper of the underlying C++ ExtensionArray. I'd appreciate some guidance at this point on how to achieve this. /cc @pitrou who looks like he may have worked on this last in https://github.com/apache/arrow/pull/4532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
