[
https://issues.apache.org/jira/browse/ARROW-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361650#comment-17361650
]
Jim Pivarski commented on ARROW-13040:
--------------------------------------
Oh, I didn't realize that.
I would be in favor of a `to_numpy_dtype`, since I have been using
`to_pandas_dtype` for exactly that purpose. (That's how I ran into this and
didn't think of checking to see if Pandas had yet another way of handling dates
and times.)
> [Python] to_pandas_dtype values are wrong or unimplemented for date-time types
> ------------------------------------------------------------------------------
>
> Key: ARROW-13040
> URL: https://issues.apache.org/jira/browse/ARROW-13040
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 4.0.0
> Reporter: Jim Pivarski
> Priority: Major
>
> Most of them mistakenly assume nanoseconds, but some are not implemented.
> Here's the complete run-down:
> {{date32/date64/time32/time64}}
> {{---------------------------}}
> {{>>> pyarrow.date32()}}
> {{DataType(date32[day])}}
> {{>>> pyarrow.date32().to_pandas_dtype()}}
> {{dtype('<M8[ns]')}}
> {{>>> pyarrow.date64()}}
> {{DataType(date64[ms])}}
> {{>>> pyarrow.date64().to_pandas_dtype()}}
> {{dtype('<M8[ns]')}}
> {{>>> pyarrow.time32("s")}}
> {{Time32Type(time32[s])}}
> {{>>> pyarrow.time32("s").to_pandas_dtype()}}
> {{Traceback (most recent call last):}}
> {{ File "<stdin>", line 1, in <module>}}
> {{ File "pyarrow/types.pxi", line 200, in
> pyarrow.lib.DataType.to_pandas_dtype}}
> {{NotImplementedError: time32[s]}}
> {{>>> pyarrow.time32("ms")}}
> {{Time32Type(time32[ms])}}
> {{>>> pyarrow.time32("ms").to_pandas_dtype()}}
> {{Traceback (most recent call last):}}
> {{ File "<stdin>", line 1, in <module>}}
> {{ File "pyarrow/types.pxi", line 200, in
> pyarrow.lib.DataType.to_pandas_dtype}}
> {{NotImplementedError: time32[ms]}}
> {{>>> pyarrow.time64("us")}}
> {{Time64Type(time64[us])}}
> {{>>> pyarrow.time64("us").to_pandas_dtype()}}
> {{Traceback (most recent call last):}}
> {{ File "<stdin>", line 1, in <module>}}
> {{ File "pyarrow/types.pxi", line 200, in
> pyarrow.lib.DataType.to_pandas_dtype}}
> {{NotImplementedError: time64[us]}}
> {{>>> pyarrow.time64("ns")}}
> {{Time64Type(time64[ns])}}
> {{>>> pyarrow.time64("ns").to_pandas_dtype()}}
> {{Traceback (most recent call last):}}
> {{ File "<stdin>", line 1, in <module>}}
> {{ File "pyarrow/types.pxi", line 200, in
> pyarrow.lib.DataType.to_pandas_dtype}}
> {{NotImplementedError: time64[ns]}}
> {{timestamp}}
> {{---------}}
> {{>>> pyarrow.timestamp("s")}}
> {{TimestampType(timestamp[s])}}
> {{>>> pyarrow.timestamp("s").to_pandas_dtype()}}
> {{dtype('<M8[ns]')}}
> {{>>> pyarrow.timestamp("ms")}}
> {{TimestampType(timestamp[ms])}}
> {{>>> pyarrow.timestamp("ms").to_pandas_dtype()}}
> {{dtype('<M8[ns]')}}
> {{>>> pyarrow.timestamp("us")}}
> {{TimestampType(timestamp[us])}}
> {{>>> pyarrow.timestamp("us").to_pandas_dtype()}}
> {{dtype('<M8[ns]')}}
> {{>>> pyarrow.timestamp("ns")}}
> {{TimestampType(timestamp[ns])}}
> {{>>> pyarrow.timestamp("ns").to_pandas_dtype()}}
> {{dtype('<M8[ns]')}}
> {{duration}}
> {{--------}}
> {{>>> pyarrow.duration("s")}}
> {{DurationType(duration[s])}}
> {{>>> pyarrow.duration("s").to_pandas_dtype()}}
> {{dtype('<m8[ns]')}}
> {{>>> pyarrow.duration("ms")}}
> {{DurationType(duration[ms])}}
> {{>>> pyarrow.duration("ms").to_pandas_dtype()}}
> {{dtype('<m8[ns]')}}
> {{>>> pyarrow.duration("us")}}
> {{DurationType(duration[us])}}
> {{>>> pyarrow.duration("us").to_pandas_dtype()}}
> {{dtype('<m8[ns]')}}
> {{>>> pyarrow.duration("ns")}}
> {{DurationType(duration[ns])}}
> {{>>> pyarrow.duration("ns").to_pandas_dtype()}}
> {{dtype('<m8[ns]')}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
