This is an automated email from the ASF dual-hosted git repository.
jorisvandenbossche pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 6b93c4a0e8 GH-38341: [Python] Remove usage of pandas internals
DatetimeTZBlock (#38321)
6b93c4a0e8 is described below
commit 6b93c4a0e8cb5110c6c4d3746f4e8bb0a8b76ec8
Author: Joris Van den Bossche <[email protected]>
AuthorDate: Mon Jan 8 14:21:10 2024 +0100
GH-38341: [Python] Remove usage of pandas internals DatetimeTZBlock (#38321)
### Rationale for this change
This usage probably stems from a long time ago that it was required to
specify the Block type, but nowadays it's good enough to just specify the
dtype, and thus cutting down on our usage of internal pandas objects.
Part of #35081
* Closes: #38341
Authored-by: Joris Van den Bossche <[email protected]>
Signed-off-by: Joris Van den Bossche <[email protected]>
---
python/pyarrow/pandas_compat.py | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/python/pyarrow/pandas_compat.py b/python/pyarrow/pandas_compat.py
index 80e313be02..3757d81a47 100644
--- a/python/pyarrow/pandas_compat.py
+++ b/python/pyarrow/pandas_compat.py
@@ -717,9 +717,15 @@ def _reconstruct_block(item, columns=None,
extension_columns=None):
elif 'timezone' in item:
unit, _ = np.datetime_data(block_arr.dtype)
dtype = make_datetimetz(unit, item['timezone'])
- block = _int.make_block(block_arr, placement=placement,
- klass=_int.DatetimeTZBlock,
- dtype=dtype)
+ if _pandas_api.is_ge_v21():
+ pd_arr = _pandas_api.pd.array(
+ block_arr.view("int64"), dtype=dtype, copy=False
+ )
+ block = _int.make_block(pd_arr, placement=placement)
+ else:
+ block = _int.make_block(block_arr, placement=placement,
+ klass=_int.DatetimeTZBlock,
+ dtype=dtype)
elif 'py_array' in item:
# create ExtensionBlock
arr = item['py_array']
