lmocsi opened a new issue, #37071:
URL: https://github.com/apache/arrow/issues/37071
### Describe the enhancement requested
Right now, ds.dataset() can include the partition key in the result if
partitioning='hive' option is used.
However, this partition key column is returned as a str, though it used to
be a date.
Would be nice, if it was returned as a datetime column, if it is in the
standard ISO format.
```
import pyarrow.dataset as ds
mytab = ds.dataset("my_table_name", partitioning='hive').to_table()
print(mytab)
```
Calendar_date is the partitioning column.
Results:
pyarrow.Table
USER_ID: double
TRX_CNT: double
DATE_OF_BIRTH: timestamp[ns]
**CALENDAR_DATE: string**
----
USER_ID:
[[1000,1001,1002,1003],[1000,1001,1002,1005],[1000,1001,1003,1005,1008]]
TRX_CNT: [[434,11,3,555],[111,32,1,2],[434,21,44,111,222]]
DATE_OF_BIRTH: [[1998-12-01 23:00:00.000000000,2002-03-13
23:00:00.000000000,1975-08-31 23:00:00.000000000,1998-12-31
23:00:00.000000000],[1998-12-01 23:00:00.000000000,2002-03-13
23:00:00.000000000,1975-08-31 23:00:00.000000000,2004-06-06
22:00:00.000000000],[1998-12-01 23:00:00.000000000,2002-03-13
23:00:00.000000000,1998-12-31 23:00:00.000000000,2004-06-06
22:00:00.000000000,1988-02-27 23:00:00.000000000]]
**CALENDAR_DATE: [["2023-08-01 00:00:00","2023-08-01 00:00:00","2023-08-01
00:00:00","2023-08-01 00:00:00"],["2023-08-02 00:00:00","2023-08-02
00:00:00","2023-08-02 00:00:00","2023-08-02 00:00:00"],["2023-08-03
00:00:00","2023-08-03 00:00:00","2023-08-03 00:00:00","2023-08-03
00:00:00","2023-08-03 00:00:00"]]**
[my_table_name.zip](https://github.com/apache/arrow/files/12296610/my_table_name.zip)
### Component(s)
Parquet, Python
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
