dvictori opened a new issue, #37813:
URL: https://github.com/apache/arrow/issues/37813
### Describe the bug, including details regarding any error messages,
version, and platform.
I expected that the output from `read_delim_arrow` and `open_dataset` on the
same CSV file should be equal. But empty string in the CSV file is being
treated differently.
Here goes a reprex.
``` r
# Arrow read_delim_arrow recognizes empty string "" as NA
# open_dataset on the same data does not
# example
library(arrow)
#>
#> Attaching package: 'arrow'
#> The following object is masked from 'package:utils':
#>
#> timestamp
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
df <- data.frame(text = c('one', 'two', '', 'four'),
n = 1:4)
tfile <- tempfile()
write.csv(df, tfile,
row.names = FALSE)
arrow_csv <- read_csv_arrow(tfile, as_data_frame = FALSE)
arrow_csv |> collect()
#> # A tibble: 4 × 2
#> text n
#> <chr> <int>
#> 1 one 1
#> 2 two 2
#> 3 <NA> 3
#> 4 four 4
arrow_dataset <- open_dataset(tfile, format = 'csv')
arrow_dataset |> collect()
#> # A tibble: 4 × 2
#> text n
#> <chr> <int>
#> 1 "one" 1
#> 2 "two" 2
#> 3 "" 3
#> 4 "four" 4
sessionInfo()
#> R version 4.3.1 (2023-06-16)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 20.04.6 LTS
#>
#> Matrix products: default
#> BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3;
LAPACK version 3.9.0
#>
#> locale:
#> [1] LC_CTYPE=pt_BR.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=C
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> time zone: America/Sao_Paulo
#> tzcode source: system (glibc)
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] dplyr_1.1.2 arrow_12.0.1.1
#>
#> loaded via a namespace (and not attached):
#> [1] vctrs_0.6.3 cli_3.6.1 knitr_1.43 rlang_1.1.1
#> [5] xfun_0.39 purrr_1.0.1 styler_1.10.1 generics_0.1.3
#> [9] assertthat_0.2.1 glue_1.6.2 bit_4.0.5
htmltools_0.5.5
#> [13] fansi_1.0.4 rmarkdown_2.23 R.cache_0.16.0 tibble_3.2.1
#> [17] evaluate_0.21 fastmap_1.1.1 yaml_2.3.7
lifecycle_1.0.3
#> [21] compiler_4.3.1 fs_1.6.2 pkgconfig_2.0.3
rstudioapi_0.15.0
#> [25] R.oo_1.25.0 R.utils_2.12.2 digest_0.6.33 R6_2.5.1
#> [29] utf8_1.2.3 tidyselect_1.2.0 reprex_2.0.2 pillar_1.9.0
#> [33] magrittr_2.0.3 R.methodsS3_1.8.2 tools_4.3.1 withr_2.5.0
#> [37] bit64_4.0.5
```
<sup>Created on 2023-09-20 with [reprex
v2.0.2](https://reprex.tidyverse.org)</sup>
### Component(s)
R
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
