randolf-scholz commented on issue #48003: URL: https://github.com/apache/arrow/issues/48003#issuecomment-3542991965
With a dedicated kernel, one should also consider some edge cases and decide whether the computation should align with `numpy`'s precedence: - `numpy` seems to round towards zero for this operation (e.g. `-1.8 -> -1`), whereas @rok 's suggestion rounds ties towards the next even value. - How should out-of-bounds values be handled? `numpy` converts these to the special `NaT` value, which is like `NaN` for durations and timestamps, but semantically different from `pyarrow`'s `null` (missing values), although these do seem to get converted to null when importing such an array from numpy: ```python import numpy as np import pyarrow as pa x = np.array([0, +10, +100], dtype="timedelta64[s]") y = x * 10e16 print(y) # [0, 10e18, 'NaT'] print(pa.array(y)) # [0, 10e18, null] ``` Trying to perform this computation directly in `pyarrow` using @rok 's suggestion results in `ArrowInvalid: Float value 10000000000000000000.000000 was truncated converting to int64`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
