jorisvandenbossche opened a new issue, #36096:
URL: https://github.com/apache/arrow/issues/36096

   When you have a arrow type that maps to a pandas extension dtype 
implementing `__from_arrow__`, the `ChunkedArray.to_pandas` conversion will 
call the `pd_dtype.__from_arrow__`, but the `Array.to_pandas` version does not.
   
   Typically, if only encounter such dtypes this if you have an pyarrow 
ExtensionArray as well, and in `ExtensionArray.to_pandas`, we also check for 
the dtype having `__from_arrow__`:
   
   
https://github.com/apache/arrow/blob/475b5b9463b64bec4e03a47e3277076db246bd35/python/pyarrow/array.pxi#L3094-L3104
   
   But the base class `Array.to_pandas` doesn't do this. And recently, one of 
the pandas dtypes that map to a non-extension array on the pyarrow side (i.e. 
DatetimeTZDtype, mapping to timestamp with tz) added `__from_arrow__`. Which 
means that for this dtype, the conversion takes a different code path. 
   
   ```
   from datetime import datetime
   import pyarrow as pa
   
   arr = pa.array([datetime(1, 1, 1)], pa.timestamp("s", tz="America/New_York"))
   table = pa.table({'a': arr})
   # doesn't call DatetimeTZDtype.__from_arrow__
   arr.to_pandas()
   # ChunkedArray does call that
   table["a"].to_pandas()
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to