[
https://issues.apache.org/jira/browse/ARROW-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17489549#comment-17489549
]
SHIMA Tatsuya edited comment on ARROW-15599 at 2/9/22, 1:30 PM:
----------------------------------------------------------------
pyarrow has aliases for each timestamp type without time zone, which can be
specified as a string.
[https://github.com/apache/arrow/blob/d59dbbc36c7950e58332d081d47c2d43ea898215/python/pyarrow/types.pxi#L2761-L2764]
If the current policy is to emphasize compatibility with `readr::read_csv`,
then ideally, the col_types "T" option would be great if one of the these
options or one of timestamp with timezone types was automatically applied.
was (Author: JIRAUSER280211):
pyarrow has aliases for each timestamp type without time zone, which can be
specified as a string.
[https://github.com/apache/arrow/blob/d59dbbc36c7950e58332d081d47c2d43ea898215/python/pyarrow/types.pxi#L2761-L2764]
If the current policy is to emphasize compatibility with `readr::read_csv`,
then ideally, the col_types "T" option would be great if one of the these
options was automatically applied.
> [R] can't explicitly convert a column as a sub-seconds typestamp from CSV (or
> other delimited) file
> ---------------------------------------------------------------------------------------------------
>
> Key: ARROW-15599
> URL: https://issues.apache.org/jira/browse/ARROW-15599
> Project: Apache Arrow
> Issue Type: Bug
> Affects Versions: 6.0.1
> Environment: R version 4.1.2 (2021-11-01)
> Platform: x86_64-pc-linux-gnu (64-bit)
> Running under: Ubuntu 20.04.3 LTS
> Reporter: SHIMA Tatsuya
> Priority: Major
>
> I tried to read the csv column type as timestamp, but I could only get it to
> work well when `col_types` was not specified.
> I'm sorry if I missed something and this is the expected behavior. (It would
> be great if you could add an example with `col_types` in the documentation.)
> {code:r}
> library(arrow)
> #>
> #> Attaching package: 'arrow'
> #> The following object is masked from 'package:utils':
> #>
> #> timestamp
> t_string <- tibble::tibble(
> x = "2018-10-07 19:04:05.005"
> )
> write_csv_arrow(t_string, "tmp.csv")
> read_csv_arrow(
> "tmp.csv",
> as_data_frame = FALSE
> )
> #> Table
> #> 1 rows x 1 columns
> #> $x <timestamp[ns]>
> read_csv_arrow(
> "tmp.csv",
> col_names = "x",
> col_types = "?",
> skip = 1,
> as_data_frame = FALSE
> )
> #> Table
> #> 1 rows x 1 columns
> #> $x <timestamp[ns]>
> read_csv_arrow(
> "tmp.csv",
> col_names = "x",
> col_types = "T",
> skip = 1,
> as_data_frame = FALSE
> )
> #> Error: Invalid: In CSV column #0: CSV conversion error to timestamp[s]:
> invalid value '2018-10-07 19:04:05.005'
> read_csv_arrow(
> "tmp.csv",
> col_names = "x",
> col_types = "T",
> as_data_frame = FALSE,
> skip = 1,
> timestamp_parsers = "%Y-%m-%d %H:%M:%S"
> )
> #> Error: Invalid: In CSV column #0: CSV conversion error to timestamp[s]:
> invalid value '2018-10-07 19:04:05.005'
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
