Wainberg opened a new issue, #40122:
URL: https://github.com/apache/arrow/issues/40122
### Describe the bug, including details regarding any error messages,
version, and platform.
I may be missing something fundamental, but Pyarrow seems to improperly
handle the conversion from datetime.datetime and datetime.time objects with
non-UTC timezones.
**Datetimes**
This Python datetime is the turn of the millenium in UTC+1:
```python
>>> dt = datetime.datetime(2000, 1, 1, 0, 0,
tzinfo=datetime.timezone(datetime.timedelta(hours=1)))
```
Converting to pyarrow makes it 11 PM in UTC+1, which is a different time
(not just the same time in a different coordinate system):
```python
>>> a = pa.array([dt])
>>> a
<pyarrow.lib.TimestampArray object at 0x7f4d0197e3e0>
[
1999-12-31 23:00:00.000000
]
>>> a.type
TimestampType(timestamp[us, tz=+01:00])
```
When converted to R (using the Arrow C API), this becomes the following
POSIXct vector:
```R
> a
[1] "1999-12-31 23:00:00"
> attr(a, 'tzone')
[1] "+01:00"
```
which is again 11 PM in UTC+1.
So the Arrow -> R conversion is consistent, but the Python -> Arrow
conversion seems like a bug.
**Times**
Creating a pyarrow array from a datetime.time strips timezone info:
```python
>>> pa.array([datetime.time(12, 0,
tzinfo=datetime.timezone(datetime.timedelta(hours=1)))]) == \
... pa.array([datetime.time(12, 0)])
True
```
which seems to be intentional according to
https://arrow.apache.org/docs/python/generated/pyarrow.Time64Array.html:
> Localized timestamps will currently be returned as UTC (pandas’s native
representation). Timezone-naive data will be implicitly interpreted as UTC.
But again, these are two different times (not just the same time in a
different coordinate system), so they shouldn't compare equal.
### Component(s)
Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
