davlee1972 commented on issue #33357:
URL: https://github.com/apache/arrow/issues/33357#issuecomment-2050402631
timestamp_parsers should work on any type of date or timestamp type. Right
now it only applies to timestamp types..
If you schema map a CSV column to timestamp it will get converted to a
timestamp column.
If you schema map a CSV column to date you get an error that '2023-01-01' is
not a date32 even if you add '%Y-%m-%d' to the timestamp_parsers list..
The only work around is to schema map a CSV column to timestamp.. Then apply
a cast("date32") on it..
I think the best solution is to either:
A. have timestamp_parsers also parse date32/64 columns in a schema
definition.
or
B. add a date_parsers arg which will support a list of date formats to
check.. i.e. '%Y-%m-%d'
You basically can't read date text in CSV files into Arrow Date columns
right now..
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
