[
https://issues.apache.org/jira/browse/ARROW-15026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17456348#comment-17456348
]
Joris Van den Bossche commented on ARROW-15026:
-----------------------------------------------
Thanks for the report!
The conversion of a datetime.timedelta object to the underlying integer value
(how it is stored in Arrow) happens here:
https://github.com/apache/arrow/blob/53ab5a000431d97fe8b2afec40034b8287bd44be/cpp/src/arrow/python/datetime.h#L149-L153
That indeed doesn't check for overflow. We do have some utilities to do
additions / multiplications with checking for overflow, see eg
{{MultiplyWithOverflow}}.
If you are interested, a PR is certainly welcome!
> [Python] datetime.timedelta to pyarrow.duration('us') silently overflows
> ------------------------------------------------------------------------
>
> Key: ARROW-15026
> URL: https://issues.apache.org/jira/browse/ARROW-15026
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Andreas Rappold
> Priority: Major
>
>
> Hi! This reproduces the issue:
> {code:java}
> # python 3.9.9
> # pyarrow 6.0.1
> import datetime
> import pyarrow
> d = datetime.timedelta(days=-106751992, seconds=71945, microseconds=224192)
> pyarrow.scalar(d)
> # <pyarrow.DurationScalar: datetime.timedelta(days=-106751992, seconds=71945,
> microseconds=224192)>
> pyarrow.scalar(d).as_py() == d
> # True
> d2 = d - datetime.timedelta(microseconds=1)
> pyarrow.scalar(d2)
> # <pyarrow.DurationScalar: datetime.timedelta(days=106751991, seconds=14454,
> microseconds=775807)>
> pyarrow.scalar(d2).as_py() == d2
> # False{code}
> Other conversions (e.g. to int*) raise an exception instead. I didn't check
> if duration overflows for too large timedeltas. If its easy to fix, point me
> in the right direction and I try to create a PR. Thanks
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
