thisisnic commented on issue #38358:
URL: https://github.com/apache/arrow/issues/38358#issuecomment-1781422749
Thanks for reporting this @dvictori! I think that what's happened here is
that `NA` is a logical (you can see by calling `class(NA)`, which is what's
tripping us up here. I can get the code to successfully run by swapping in
`NA_character_`:
``` r
library(arrow)
library(dplyr)
df <- data.frame(i = 1:4,
date = c(as.Date('2013-01-01'),
as.Date('2013-01-02'),
as.Date('2033-01-03'),
as.Date('2013-01-04')))
tfile <- tempfile()
write.csv(df, tfile,
row.names = FALSE)
arrow_dataset <- open_dataset(tfile, format = 'csv')
# column type is date32[day]
arrow_dataset
#> FileSystemDataset with 1 csv file
#> i: int64
#> date: date32[day]
arrow_dataset |>
mutate(date = if_else(date > '2014-01-01', NA_character_, date))
#> FileSystemDataset (query)
#> i: int64
#> date: date32[day] (if_else((date > 2014-01-01), null[date32[day]], date))
#>
#> See $.data for the source Arrow object
arrow_dataset |>
mutate(date = if_else(date > '2014-01-01', NA_character_, date)) |>
collect()
#> # A tibble: 4 × 2
#> i date
#> <int> <date>
#> 1 1 2013-01-01
#> 2 2 2013-01-02
#> 3 3 NA
#> 4 4 2013-01-04
```
I'm wondering if this might be something we can add a check for in the R
code though, and automatically change the type of `NA` if necessary in our
binding to `if_else()` as it's a weird thing we wouldn't necessarily expect
folks to just know.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
