jaychia opened a new issue, #37580:
URL: https://github.com/apache/arrow/issues/37580

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   PyArrow version: 13.0.0
   Python version: 3.10.9
   OS: MacOSX, arm64
   
   Some Timestamp values cannot be successfully round-tripped through 
PyArrow-Parquet[INT96 types]-PyArrow, even though they are within the 
expressible range of the timeunit ("milliseconds" in the example below).
   
   ---
   
   To reproduce:
   
   ```
   In [1]: import datetime
   
   In [2]: import pyarrow as pa
   
   In [3]: import pyarrow.parquet as papq
   
   In [4]: ts = pa.array([datetime.datetime(1200, 1, 1, 0, 0, 59), 
datetime.datetime(2000, 1, 1), datetime.datetime(2400, 1, 1)], 
pa.timestamp("ms"))
   
   In [5]: table = pa.table({"ts": ts})
   
   In [6]: papq.write_table(table, "test.pq", 
use_deprecated_int96_timestamps=True)
   
   In [7]: papq.read_table("test.pq", coerce_int96_timestamp_unit="ms")
   Out[7]: 
   pyarrow.Table
   ts: timestamp[ms]
   ----
   ts: [[1784-07-20 23:35:32.709,2000-01-01 00:00:00.000,2400-01-01 
00:00:00.000]]
   ```
   
   Note that the first timestamp is received as `1784-07-20 23:35:32.709` 
instead of the expected `1200-01-01 00:00:59.000`.
   
   However, if we change the timestamp from `datetime.datetime(1200, 1, 1, 0, 
0, 59)` to `datetime.datetime(1200, 1, 1, 0, 0, 00)` it now seems to work.
   
   ```
   In [9]: ts = pa.array([datetime.datetime(1200, 1, 1, 0, 0, 00), 
datetime.datetime(2000, 1, 1), datetime.datetime(2400, 1, 1)], 
pa.timestamp("ms"))
   
   In [10]: table = pa.table({"ts": ts})
   
   In [11]: papq.write_table(table, "test.pq", 
use_deprecated_int96_timestamps=True)
   
   In [12]: papq.read_table("test.pq", coerce_int96_timestamp_unit="ms")
   Out[12]: 
   pyarrow.Table
   ts: timestamp[ms]
   ----
   ts: [[1200-01-01 00:00:00.000,2000-01-01 00:00:00.000,2400-01-01 
00:00:00.000]]
   ```
   
   ### Component(s)
   
   C++, Parquet


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to