On Fri, 3 Dec 2021 at 23:36, Wenyi Huang <[email protected]> wrote:
>
> Hi Arrow Team,
>
> What is the best data type to save Time of the Day if I want to use Pandas
> Datafrome, but dump data to parquet (or other formats) via PyArrow?
>
> I see that pandas Arrow conversion does not convert datetime.time nor
> timedelta.
Can you show an example? Because I think that this conversion should
handle both cases:
>>> import datetime
>>> df = pd.DataFrame({"time": [datetime.time(9)], "timedelta":
>>> [pd.Timedelta("9 hours")]})
>>> df
time timedelta
0 09:00:00 0 days 09:00:00
>>> pa.table(df)
pyarrow.Table
time: time64[us]
timedelta: duration[ns]
----
time: [[09:00:00.000000]]
timedelta: [[32400000000000]]
The resulting Arrow table has columns with time and duration type
(duration is the Arrow equivalent for timedelta).
Joris
>
> The use case is that I want to save the time of the day column. So that while
> reading (parquet or other formats), I can filter by time.
>
> Best,
> Wenyi
>
> --
> Wenyi Huang
> LinkedIn: https://www.linkedin.com/in/harrywy/
> Google Scholar: https://scholar.google.com/citations?user=K-RWg7gAAAAJ&hl=en
> Email: [email protected]
>
>
